Applies to: Windows 7, Windows 8, Windows 8.1
When addressing application compatibility issues in preparation for a deployment of Windows, among the most flexible and powerful tools available are application compatibility fixes, commonly called shims. However, many organizations do not leverage shims to the extent that they could, either because they don’t understand the underlying technology or because they do not have a process in place to manage shims over the remaining lifetime of the applications they are used to remediate.
This article shares best practices for addressing application compatibility issues using shims. It details how shims work, when to consider applying shims, and how to manage the shims you do apply.
In this article:
Application compatibility in Microsoft Windows operating systems is one of the fundamental pillars of its development, alongside performance, reliability, and manageability. To reduce deployment costs and accelerate adoption, Microsoft invests in deep technical solutions to ensure broad compatibility of existing software, driving compatibility into the engineering and release process.
The Microsoft Windows Application Compatibility Infrastructure (Shim Infrastructure) is one such powerful technical solution. As the Windows operating system evolves – changing to support new technology, incorporate bug fixes, and/or implement a modification in strategy – changes to the implementation of some Windows Application Programming Interface (API) functions may affect applications that use them. Because the nature of the dependencies on particular implementations vary so much, it may not be possible to create a single implementation that works for all existing software, while simultaneously providing the desired new behavior. In the past, we would address such compatibility issues by placing branches within the source code for Windows, but doing so presents a long-term challenge for the serviceability and reliability of the Windows operating system. It also means that we have to be aware of the issue before we ship Windows – and that is certainly not always true for enterprise software! Using the Shim Infrastructure, however, you can target a specific application fix specifically towards a particular application (and typically, for particular versions of that application), with these fixes housed outside the core Windows functions and maintained separately.
The Shim Infrastructure implements a form of API hooking. Specifically, it leverages the implementation of dynamic linking to redirect API calls from Windows itself to alternative code – the shim itself. The Windows Portable Executable (PE) and Common Object File Format (COFF) Specification includes several headers, and the data directories in this header provide a layer of indirection between the application and the linked file. Calls to external binary files take place through the Import Address Table (IAT). Consequently, a call into Windows looks like Figure 1 to the system.
Figure 1. Application calling into Windows through the IAT
Specifically, you can modify the address of the Windows function resolved in the import table, and then replace it with a pointer to a function in the alternate shim code, as shown in Figure 2.
Figure 2. Application redirected to the shim prior to calling Windows
This indirection happens for statically linked dll files when the shimmed application is first loaded. Windows also shims dynamically linked dll files by hooking the GetProcAddress API.
You may find certain consequences of the Shim Infrastructure design relevant when determining your shim strategy.
First, as shown in Figure 2, the code that runs inside a shim still sits outside Windows. Consequently, Windows holds shim code to the same security restrictions as the application code itself. In fact, to Windows, the shim code appears to be application code. As a result, you cannot use shims to bypass any security mechanisms present in Windows. For example, no shim is available to bypass the User Account Control (UAC) prompts yet still run the application with elevated permissions (a common request from enterprise customers first experimenting with shims in an attempt to remove administrator rights from more users). You can shim the application not to require administrator rights, or you can shim it to prompt the user to provide administrator rights, but in order to receive administrator rights with UAC enabled, the user will have to approve the elevation. The same is true for code that you write yourself.
Therefore, when evaluating the security implications of using shims in your enterprise, you are not opening any additional security vulnerability. In fact, using shims to avoid having to loosen security descriptors or make security policy more lax can frequently be the more secure choice. For example, without shims, you may be able to mitigate a compatibility issue by loosening the ACLs on a particular directory, but this decision has an effect on the entire system. Using shims, you may be able to redirect the file access to a per-user location for that application. As another example, an application may be explicitly checking for administrator rights. Without shims, you may have to grant the application administrator rights to pass this check. If the check is an unnecessary one, however, a shim could simply lie about whether the current user has administrator rights, allowing the check to succeed without exposing additional security surface area.
Because the Shim Infrastructure, in essence, injects additional code into the application before it calls into Windows, any mitigation you can use a shim to accomplish can be done by modifying the application code itself. At a minimum, the application could include code similar to what Windows implements in the shim immediately prior to calling into Windows APIs. As a result, even when shims are not used as the final remediation, they can often be very instructive during troubleshooting.
The performance impact of shims will vary, depending on what the shim is doing. Let’s consider some examples. A version lie shim, which simply pretends to be an earlier version of Windows, performs about the same– it doesn’t take any more work to return a value of 5.1 than it does to return a value of 6.2. The ForceAdminAccess shim, as another example, performs much faster than the APIs it intercepts – since it always says “yes, the user is an admin” without looking, it avoids the performance impact of actually determining if the user is an administrator! As a final example, the CorrectFilePaths shim inspects the path to determine if it should be modified, so you add some performance overhead to do the string comparison for all intercepted file system calls, plus the additional overhead of replacing the string argument when a match is found. As you can see, there is not one easy answer, so it’s best to test when you have performance requirements. What most customers find, however, is that the applications that require shims for remediation are often older, and as a result still meet their performance criteria by virtue of the fact that today’s hardware is much faster than the hardware it ran on years ago when they acquired the software.
Finally, because shims run as user-mode code inside a user-mode application process, you cannot use a shim to fix kernel-mode code. For example, you cannot use shims to resolve compatibility issues with device drivers or with other kernel-mode code. (For example, some antivirus, firewall, and antispyware code runs in kernel mode.)
Back to top
When Microsoft consultants work with customers on using shims to resolve compatibility issues with Windows, they begin the conversation with an overview of how shims work, ensuring that people at various levels in the organization understand the technical implications of using shims for mitigations. However, the decision is more than a technical one.
When you are considering whether shims are appropriate, your first question should be whether your business will require a valid support agreement for the application on the version of Windows you are seeking to deploy. If the answer is yes, then your first check should be whether you have a version that is supported by the vendor. If you have a vendor supported application that fails, you don’t need to use shims – you just need to contact the vendor and open a support case to get it fixed!
It is when vendor support is not required that most customers consider using shims. You may be taking a version that is not supported on the version of Windows you are trying to deploy, and which has a failure, and attempting to get it working again. But keep in mind that shimming the application to get it working doesn’t change the support state – it will remain unsupported by the vendor even after you have gotten it working.
The scenarios in which Microsoft consultants have assisted customers in using shims to mitigate application issues include:
In general, Microsoft consultants have found that good communication and collaboration among technology owners (“do the shims succeed in making the application sufficiently compatible”) and business owners (“can I accept the support terms of using a shimmed version of this incompatible application”) helps the decision process.
While a thorough description of applying a shim to a particular application is outside the scope of this white paper, one aspect that is important to note is that shims can be applied to particular versions of applications, either as “up to or including” or just a particular version. Either one ensures that the next version of the application that is released will no longer have the shim applied.
This is important to many customers. They want to ensure that they can fix the incompatible application for a particular version but still encourage the developer (whether an internal development team or a vendor) to fix the application by having the shim no longer apply the next time the version number is incremented.
While support policies for applications made compatible with Windows using shims is up to each software vendor, another frequent question is, how is the code for the shims themselves supported?
Shims ship as part of Windows and are updated through Windows Update. Consequently, they fall under the same support terms as the rest of the Windows operating system.
You can apply shims that the Windows product team creates to your own applications, but Microsoft does not provide the necessary tools for you to create your own custom shims leveraging the shim infrastructure.
Back to top
If you have decided to use shims as part of your application compatibility mitigation strategy (for certain classifications of applications), the next question is, which strategy should you employ for managing custom shim databases? The customers Microsoft consultants have worked with have tended to select one of two approaches to managing their custom shim databases: deploying fixes as part of the application package or managing a centralized custom shim database.
Regardless of the approach your organization chooses, here are general recommendations for improving the management of custom shim databases:
One strategy for deploying application fixes is to include the custom shim database—containing a single entry for the application the package is installing—directly into the installation package. During the early phases of compatibility testing, this can seem like the easiest approach. However, over time this approach can grow more complex. Microsoft consultants recommend evaluating the following considerations prior to selecting this approach.
The thing to keep in mind is that custom shim databases are still databases. Consequently, if you were to have 1,000 shimmed applications, it takes longer to open and query 1,000 different one-row databases looking for a match to a given application than it would to open a single database and query against 1,000 rows. Note that this performance penalty is something you pay every time a new process is created on Windows. While not generally significant enough for end users to notice, depending on the environment (because of the law of large numbers) this can eventually average out to an enterprise-wide increase of higher power consumption that is no longer trivial in nature.
It is possible that you will eventually find that the shims assigned to resolve a set of compatibility issues in an application are not comprehensive and that later you will need to deploy an updated version of the custom shim database that resolves the additional issues your organization later discovered. If you deployed the original custom shim database as part of the installation package, you will need to locate each client that has installed this application and the original custom shim database for it to replace it with the new version. While this strategy can be a good approach if you are shimming only a few applications, when the number of applications grows beyond a few, most customers eventually opt against using it.
An alternate strategy most customers consider (and most end up using) is to manage either a single custom shim database or several custom shim databases for large subsets of the organization. Doing so makes it easier to enforce policy and provide consistent updates to application mitigations you discover that you need to support your migration to Windows 7. Microsoft consultants recommend evaluating the following considerations prior to selecting this approach.
If you are planning to manage a centralized custom shim database, ensure that the tools are in place to deploy and update this custom shim database across all the computers in your organization that require this. As additional applications have shims applied to them, ensure that the target computers have the updated shim database installed prior to using the application.
If you are taking the centralized approach, make sure that you have identified appropriate owners and that the application owners and testers have a clear path for escalating a request for a shim to result in deployment of an update to target computers.
Microsoft consultants have found that this strategy tends to be the best approach, when you have a solid deployment infrastructure in place and centralized ownership of the process. The primary advantages have been accountability and simplifying support (as the deployment of a particular version of a shim is more consistent across the organization).
Back to top
Customers who have selected a centralized custom shim database approach benefit from the improved performance of searching a single database to determine whether Windows should apply a shim to a particular executable file. A frequent question Microsoft consultants receive is how to merge custom shim databases to create a single custom shim database. Customers have generally taken the following approach:
Deploying a custom shim database to users requires the following two actions:
While any approach that completes these two actions will work, customers commonly use one of the following two approaches:
Note that you must ensure that the installation of the custom shim database executes with administrative rights.
This example script is taken from the custom action of a Windows Installer (MSI-based) installation of a custom shim database:
Because testing and mitigation of application compatibility issues typically happens prior to the deployment of a new version of Windows, a common approach is to include the custom shim database containing all known issues at the time of deployment with the corporate image. Then, as you need to update your custom shim database, you could provide these updates using one of the mechanisms described above. This is the methodology that Microsoft uses to manage the System shim database. The initial version was released with the Release to Manufacturing (RTM) version of Windows, and updates are provided with Windows Update. When you use this approach, you are using a methodology proven at a very large scale.
Back to top
Shims are a powerful tool for mitigating application compatibility issues and moving forward with your Windows 7 deployment. However, understanding when to shim and how to manage shims within an organization can be a challenge.
This white paper has reviewed the approaches that other customers have used to leverage this powerful technology while minimizing risk and costs. Understanding how shims work is often the most important step in adopting the technology. You have seen how to set criteria for selecting the applications for which using this tool is appropriate after the decision has been made to adopt the technology. Finally, this paper discussed the approaches that most customers use to manage and deploy a custom shim database.
Back to top
Chris "The App Compat Guy" Jackson is a Principal Consultant and the worldwide lead for application compatibility at Microsoft, specializing in Windows, Internet Explorer, and Office internals. Jackson is a widely recognized expert in the field of application compatibility, creating technical documentation, training, and service offerings used inside and outside of Microsoft and based on years of real-world experience with enterprise customers and independent software vendors. Author or co-author of numerous technical papers and articles, he is also a featured speaker at major industry conferences around the world and publishes a popular blog.