Windows ConfidentialKeep Your Caches Close
Raymond Chen
Caches can improve performance, but you have to use them correctly. There are usually a lot of parameters you can fiddle with, but even if your cache's policies are tuned correctly, you still have to use the cache correctly.
Here's an example based on actual events. I've transposed the scenario into something readers will more readily understand so I don't have to go on about the details of a technology that's irrelevant to the underlying issue.
Suppose a New York-based company has a central Web server that functions as a document archive. The company can make requests such as "Give me the Petition for Dismissal from the Johnson case that we filed on June 23, 2005." This is effectively a static Web page since that document never changes—if the petition needs to be updated, an amendment is filed while the original document remains unchanged because, well, they already filed it. To change the original petition they'd need a time machine to go back in time. (And while they're at it, they'd accidentally change the course of human history through some seemingly insignificant action.)
This usage pattern maps well to a caching Web proxy server. If somebody asks for the June 23 Petition and then a short time later somebody else asks for the same document, the proxy server can respond to the second query and provide the same results as the first query, saving a round-trip to the central server.
This particular organization already had a caching Web proxy server and configured the Miami office to take advantage of it. But they found no performance improvement. Even when requesting the same document several times in a row from the proxy, the response was just as slow as requesting the document directly from the central server. What went wrong? Was the proxy server broken?
The proxy server was, in fact, working just fine. The problem was where the proxy server was located: the Miami office had a slow connection to New York and was using the caching Web proxy that was installed at the New York office.
The diagram in Figure 1 makes it pretty clear why the proxy didn't speed up anything. Even if the proxy did a perfect job and every request could be filled without going to the central server, the documents still had to be transferred over the slow connection between Miami and New York.
Figure 1 Proxy server on the wrong side of a slow connection (Click the image for a larger view)
If you're going to use a cache, the cache needs to be faster than the thing it's caching. In this example, the connection from the Miami office to the proxy server needs to be faster than the connection from the Miami office to the central server. The proxy server needs to be installed closer to the Miami office. For the best results, the proxy server should be installed in Miami, as shown in Figure 2.
Figure 2 A far more useful proxy setup for the Miami office (Click the image for a larger view)
With the proxy server located in Miami, when an attorney in Miami wants to call up the June 23 Petition, the request will go to the local proxy, which will quickly hand over the document from its cache. If the document isn't already in the cache, the document will be retrieved from the server and then cached for the next time someone in the Miami office requests it.
Caches take advantage of temporal locality, but moving the proxy to the Miami office also exploits geographic locality: the attorneys in the Miami office tend to request different documents than the attorneys in the New York office (since, of course, they handle different cases), and keeping a different cache for each remote office allows each one to serve its assigned office better.
The organization installed a proxy server in the Miami office and configured the computers in that office to use the local proxy. Now they're happy with the performance.
Raymond Chen's Web site, The Old New Thing, and his identically titled book (Addison-Wesley, 2007) both deal with Windows history and Win32 programming. Some club soda will get that stain out.