Memory Performance Testing

Published: November 1, 1998

By Jackie Barrera, Product Marketing, Kingston Technology Company

A Quantitative Demonstration of the Benefits of More Memory for Windows NT Servers

On This Page

1. Executive Summary
2. Scope of Work
3. What We Found
4. Mindcraft Certification
5. About Mindcraft
6. About Kingston

1. Executive Summary

Kingston Technology Co. contracted Mindcraft, an independent testing lab, to perform a quantitative performance study that answered the following questions:

1. How much does Web server performance improve as you add more memory?

   The essential graphic to answer these questions is a chart showing Web server operations per second as a function of the amount of memory. In another chart, we show the effect the amount of memory has on response time.

2. How much more can an application server do with additional memory?

   We answer this question in a chart showing the improvement of application server performance as a function of the amount of memory in the server.

3. How much can a LDAP (Lightweight Directory Access Protocol) directory server do with more memory?

   We chart the results by showing performance as a function of memory size.

2. Scope of Work

The goal of this benchmark was to show the performance benefits of more memory in Windows NT-based servers. We answer the following questions for the applications we tested:

• How much more work can an application do with each memory configuration?

• When is it better to add memory to a system than to add processors?

• How many users can the system support with this much memory?

Server Computer

For testing purposes, we selected a Compaq ProLiant 5000. This system uses 200MHz Pentium Pro CPUs, and can be configured with one to four processors. We tested the system with 64MB of memory, and we expanded up to 1GB of memory.

• Web Server Testing

  Mindcraft used SPECweb96 to test Web server performance. The file working size was 622MB. We used Microsoft's IIS (Internet Information Server) 4.0 Web server software.

• Application Server Testing

  Mindcraft used the Ziff-Davis ServerBench benchmark to test application server performance as a function of memory size.

  Mindcraft set up the machines to use Windows NT 4.0 as the client operating system. They used 24 client systems simulating a total of 72 clients.

• Directory Server Testing

  Mindcraft used DirectoryMark to test the performance of an LDAP (Lightweight Directory Access Protocol) directory server test.

  Mindcraft used a directory size of 50,000, 200,000 entries, and 400,000 entries

3. What We Found

Web Server

• Double your memory, cut your response time by more than half.

  A web server houses HTML files that transfer to a client in response to HTTP requests. It can also use applications to generate HTML code. The bottom line is, people don't want to wait around for a slow web page. While performing this benchmark test we found that by increasing the system's memory from 128MB to 256MB, with the system running 1CPU, a user can cut the server response time by 63%. Also, by doubling memory from 256MB to 512MB, a user will improve the system response time by 59%.

  If running a server with 2 CPUs, and system memory is upgraded from 512MB to 1GB, the system will improve response time by 36%.

  By looking at these results we can say:

  Double your memory = cut your response time in half

  Specweb96 Response Time Chart

  

• Increase your server performance by over 500%

  We also measured how many operations per second can be performed on the server as the amount of RAM is increased.

  This benchmark shows that the user can increase web server performance by 540% by upgrading a system from 128MB to 512MB, and running the server with 1 CPU.

  When upgrading from 512 MB to 1GB, and running a server with 2CPUs, a user can increase performance by 51%.

  Specweb96 Operations per second

  

  It is important to notice in both charts that when running a system with 512MB, whether it is running one CPU or two CPUs, performance does not change significantly. This indicates that memory plays the key role in performance increase. These charts show that in this case increasing the amount of RAM on a server will be recommended, rather than increasing the number of CPUs.

Application Server

• Support three to ten times as many clients by increasing your memory

  We tested the application server performance by adding clients in sets of four. We started testing with 1 client, and increased it to a maximum of 72 clients accessing the server at on time.

  Tests demonstrate that increasing your memory in an application server significantly improves the application server performance.

  By increasing base memory from 128MB to 256MB the server was able to support five times as many people before the transactions per second dropped. By increasing memory from 256MB to 512MB the server was able to support ten times as many people.

  So by doubling the server memory a user can dramatically increase the number of clients supported on an application server.

  ServerBench Score

  

  This chart illustrates that the number of clients supported on a server is directly related to the amount of RAM installed in the system.

• Increase the system memory on an application server = increase the number of clients supported

  We also found out that whether running one CPU or two CPUs with 512MB, a performance drop will take place at the same point (see chart below). Once again, this shows that memory plays a very important role on application servers.

  When running two CPUs the highest performance increase happens when 1GB of RAM is added to the system. At this point a server can support the highest amount of clients without showing significant performance drop.

  

Directory Server

We tested three size databases:

• Double your memory, increase your directory server performance by an average of 1000%

  On our first scenario, a 110MB database, we found out that by increasing the base memory from 128MB to 256MB a server will experience a performance increase of 947%. This is a very dramatic performance increase for clients accessing requests from the server.

  50,000 entry database scenario (110MB database)

  

  On our second scenario, a database of 375MB, we found out that by doubling memory from 256MB to 512MB, access request time will improve by 3000%.

  200,000 entry database scenario (375MB database)

  

  Finally, on our third scenario, a database of 690MB, by doubling memory from 512MB to 1GB, system response time will experience a performance increase of 248%.

  400,000 entry database scenario (690MB database)

  

4. Mindcraft Certification

Mindcraft was commissioned by Kingston Technology Company to produce an independent and unbiased assessment of the benefits of memory on Windows NT Servers.

The tests were conducted on the second quarter of 1998.

These tests should be reproducible by others who use the same test lab configurations as well as the computer software configurations used in this benchmark test.

5. About Mindcraft

Mindcraft is a service-oriented, independent test lab. The company was founded in 1985 to provide high quality services and products to vendors and end users who want to test software, systems, and network products.

Mincraft is the largest Accredited POSIX Testing Laboratory in the world. As part of their accreditation by The National Voluntary Laboratory Accreditation Program (NVLAP, part of the United States National Institute of Standards and Technology), they have developed a rigorous quality system that meets international standards.

For more information about Mindcraft go to: https://www.mindcraft.com

6. About Kingston

Kingston Technology is the world's largest independent manufacturer of memory products for servers, workstations, desktops, portables, and electronic devices. Over the last ten years, Kingston has diversified its product lines to include processor upgrades, flash memory, networking hardware, and storage products. With strictly regulated ISO-registered-facilities in the United States (ISO 9001), Ireland (ISO 9002), and Taiwan (ISO 9002), Kingston markets its products through an extensive worldwide network of distributors, major reseller chains and independent dealers.

In August of 1996, Kingston became part of SOFTBANK Corp. SOFTBANK Holdings Inc. is the holding company for all of SOFTBANK Corporation's U.S.-based activities. Its major operating companies include Ziff-Davis, Kingston Technology Company, SOFTBANK Content Services, and UTStarcom. SOFTBANK is the largest shareholder of Yahoo! and E*Trade, as well as a minority investor in The Rights Exchange, GeoCities, CyberCash, First Virtual Holdings, and E-LOAN. In addition, through affiliated venture funds in the U.S. and Japan, the SOFTBANK Group has made more than 70 investments in Internet companies. Access the SOFTBANK website at https://www.softbank.com. Visit the Kingston home page on the Internet at https://www.kingston.com.

NOTICE:

Products and corporate names are trademarks and/or registered trademarks of their respective owners.

Kingston and/or Mindcraft shall not be liable for errors or omissions contained herein, nor for incidental or consequential damages resulting from the furnishing, performance, or use of this material.

We at Microsoft Corporation hope that the information in this work is valuable to you. Your use of the information contained in this work, however, is at your sole risk. All information in this work is provided "as -is", without any warranty, whether express or implied, of its accuracy, completeness, fitness for a particular purpose, title or non-infringement, and none of the third-party products or information mentioned in the work are authored, recommended, supported or guaranteed by Microsoft Corporation. Microsoft Corporation shall not be liable for any damages you may sustain by using this information, whether direct, indirect, special, incidental or consequential, even if it has been advised of the possibility of such damages.