Scalability of Micro Focus Enterprise Server and Microsoft Windows Server 2003

The paper analyzes the performance of mainframe style workloads on large, multiprocessor Microsoft Windows Server 2003, Datacenter Edition environments. We look at a benchmark workload that simulates much of the CPU and I/O behavior of IBM mainframe COBOL and Customer Information Control System (CICS) transaction processing environments.

This benchmark application was measured on an IBM mainframe environment and then, using the Micro Focus Lift and Shift process, moved without code modification to the Micro Focus Net Express Enterprise Server with the Mainframe Transaction Option (MTO) on 8, 12, and 16-CPU processor systems.

This study by Electronic Data Systems (EDS), Micro Focus, Microsoft, and Unisys Corporation estimates the equivalent of 1,715 mainframe millions of instructions per second (MIPS) for an 8-CPU enterprise-class system. Further, we show COBOL performance scales linearly under Microsoft Windows similar to what is expected in IT data centers.

This information applies for Microsoft Windows Server 2003.

The current version of this paper is maintained on the Web at https://www.microsoft.com/whdc/winhec/papers05.mspx.

References and resources discussed here are listed at the end of this paper. Additional resources can be found on the Mainframe Migration Alliance (MMA) Web site at https://www.mainframemigration.org.

On This Page

Introduction
Business Application Hardware Architectures
Transaction Processing Benchmarks
Mainframe Migration Architecture
Scalability Results and Analysis
Conclusion
Resources

Introduction

This paper is aimed at business professionals (CIO’s and IT managers) who manage mainframe information systems and wish to understand alternative hosting environments for those systems.

A previous publication (see Mainframe Migration, 2004 in the Resources section below) analyzed the performance of these mainframe alternative systems to process common business-oriented applications. It showed how Micro Focus Net Express Enterprise Server with the Mainframe Transaction Option (MTO), together with Microsoft Windows Server 2003 Datacenter Edition, Service Pack (SP)1, combines with high-end multiprocessor hardware to perform at a mainframe equivalent rate of 1347 mainframes millions of instructions per second (MIPS). This figure represents mainframe computing rates larger than 80 percent of IT data centers. The current, new, study by Electronic Data Systems (EDS), Micro Focus, Microsoft and Unisys Corporation confirms the earlier benchmark of 1,347 MIPS and raises it to 1,715 MIPS for a base 8-CPU enterprise-class system.

The Micro Focus Lift and Shift process of mainframe migration assures that re-hosting is performed with minimal changes to COBOL applications currently running in an IBM Customer Information Control System (CICS) transaction system, IBM DB2 database system and IBM zOS operating environment.

In addition to raw performance of any hosting platform, IT professionals are inherently interested in the scalability of such systems. The ability of these alternative mainframe systems to grow with their business growth, by adding new hardware, is termed scalability. This paper analyzes all components (operating system, transaction system, database system) under a benchmark environment designed to simulate a CICS COBOL application as demands are made to increase users and transactions on the system.

For decades, countless organizations around the globe have relied on the performance, capacity, reliability and longevity of IBM mainframes to support their businesses, by processing data, printing billing details, running internal reports, calculating insurance premiums or bank balances, driving applications run by the customer call center staff, and more. In the 21st century, three quarters of the world’s data still resides on the mainframe, accessed by COBOL systems. Information is processed as online transaction systems (OLTP), such as the IBM CICS system running on IBM zSeries mainframes. These software and hardware systems are designed to be extensible and grow with a business. They can even deliver increased capacity at year-end or other times of peak workload.

For this paper we (researchers from Micro Focus, EDS, Unisys Corporation, and Microsoft) analyze the ability of this environment to scale as workload (transactions) increase at a rate proportional to increases in hardware. We conclude that, given enterprise-class hardware, Windows Server 2003, Micro Focus Enterprise Server with MTO and database systems scale linearly as workload increases.

Business Application Hardware Architectures

Initially linked to the clock speed of a processor, MIPS ratings have been expanded to imply a certain measure of workload capacity. Machines with differing input/output (I/O) subsystems, clustering interconnections and memory configurations can generate different MIPS ratings independent of the CPU speed. Mainframe business applications require a well-balanced I/O system. Amdahl’s law has held for business applications since the mid -1960s. For every instruction executed one byte of data is moved between the disk and CPU. As we look at using Micro Focus Lift and Shift process for migrating applications off the mainframe to Windows-based environment the ratio logically continues to hold.

We are now seeing the emergence of “enterprise-class” Windows and Intel systems that are well balanced. These systems are similar to mainframes in terms of high speed network attachments, fiber optic interconnects to disk, large disk buffer caches (battery backed for transaction durability requirements), dual power supplies and bus architectures for reliability, tiered client networks and end-user stations relieving the host of low-level handling. It is this class of machines on which we focus our benchmarking efforts. These systems are very different from “personal” desktop or laptop computers where a 1000:1 ratio of CPU to I/O is not uncommon, instead of the desired 1:1 for business online transaction processing systems.

The 64-bit Intel and Windows architectures show promise to run even larger images and workloads. However, we focus on Intel XEON-based 32-bit systems. Most COBOL applications moving from the mainframe host still employ 32-bit (actually, 31-bit) designs. The ability of the operating system to manage multiple 32-bit environments and allocate memory resources that exceed the 32-bit virtual memory of any one process is important to scalability. Enterprise-class Intel hardware architectures, the Windows operating system, and Micro Focus Enterprise Server with Mainframe Transaction Option can exploit physical memory many times that of the 32-bit virtual address space and this is brought out in these scalability tests.

The enterprise-class environment used to verify scalability of the Lift and Shift environment was a Unisys ES7000 Orion. The architecture of this system allowed partitions to be set up consisting of various memory and processor configurations, just as IBM logical partitions (LPARs) or Amdahl multiple domain facility (MDF) are used to fence off workloads from one another (see following table). The mainframe migration scalability test environment consisted of partitions of 4, 8, 12 and 16 processor configurations, all but the first on the Unisys ES7000. The smaller 2-CPU partition test was conducted on a different architecture to compare smaller server machines. Although capable of 32-CPU configurations as a single system Windows image, the MIPS capacity of these Unisys images were too large to be driven by our test workload generation machine environment. The Unisys “system under test” partition required a 3x horsepower to simulate the end-user workload and gather response time measurements.

Transaction Processing Benchmarks

The transaction processing and database community utilize a series of benchmarks by the Transaction Processing Performance Council (TPC) to measure relative throughput of a system. Since 1993, the TPC-C (See TPC, 2002 in the Resources section below) online transaction processing benchmark has been used to demonstrate the effectiveness of hardware platforms, database systems and transaction systems. One problem with the benchmark has been its popularity: it is now featured in Wall Street Journal advertisements when new records are reached. Over the years this popularity has led TPC-C to become a highly tuned benchmarking vehicle to demonstrate a hardware or software vendor’s effectiveness of their product. The record-setting performances often no longer reflect real world (mainframe) workload environments. However, the roots of the TPC-C specification lie in classic mainframe data processing needs of a company that must manage, sell or distribute a product or service (such as car rental, food distribution, parts supplier, etc.).

We developed a COBOL implementation of the TPC-C business logic complete with embedded SQL statements that are handled by SQL preprocessors. In this mainframe migration environment, as on the mainframe itself, the screen interface is handled through the use of 24-line, 80-column terminal screen (3270 terminal) used for data entry and report generation. It is the responsibility of Micro Focus Enterprise Server with the Mainframe Transaction Option to provide the commit coordination to the database management system through an XA-compliant interface. The benchmark simulates data entry errors which cause transactions to fail about 1 percent of the time.

The benchmark was run on an IBM mainframe with the zOS operating system, CICS transaction system and DB2 database system. The COBOL remained unchanged when it was moved to Windows Server 2003, Micro Focus Enterprise Server with the Mainframe Transaction Option and the IBM DB2 UDB 8.1 database system. The database in the Windows environment allowed certain optimizations that are typical in IBM environments to be employed in the mainframe migration environment. For example, SQL statements removed from the COBOL source by a precompiler are entered into a DB2 package for faster execution. Database locking table size, buffer sizes and database growth characteristics were the same in both environments, as was the layout of data types, tables and index structure of both systems.

The TPC-C benchmark was chosen for these scalability tests largely because it is well documented and tested and that it models the historical trend in IBM CICS COBOL business transaction systems. IBM researchers in the IBM Systems Journal (See Characteristics, 2001 in the Resources section below) found that TPC benchmarks fall reasonably within the range of real world behavior in some areas; in other areas, they were not representative. For example, the IBM researchers found that the transactions were longer and contained fewer read-only transactions than in production workloads. They also had collisions with other transactions in the mix. TPC-C was found to have fewer transaction types and hold locks longer than production CICS workloads.

The IBM researchers found that the TPC-C differs from production CICS workloads in what they call burstiness (or large variability) of I/O activity, due to the fact that an application’s use of the I/O system varies widely. In production, there are often cases of large amounts of I/O (batch) jobs intermingled with online transaction processing workloads. The ability of mainframe systems (such as job schedulers) to take advantage of system idle time to balance a workload over a period of time is key to overall system throughput.

Despite these differences between production CICS workloads and an artificial benchmark workload such as TPC-C, we found that running the exact same COBOL application with CICS transaction APIs and SQL database statements on the mainframe and non-mainframe systems represents a good apples-to-apples comparison of the platforms. These ratios have been born out in customer workloads that have undergone migration from a mainframe.

Mainframe Migration Architecture

The architecture of the Micro Focus Enterprise Server with the Mainframe Transaction Option works well with the Windows Server 2003, Datacenter Edition to distribute transactions effectively across all available processors. This is important for the system to scale accordingly as more processors were added to the system during the scalability tests.

Figure 1. Enterprise Server with Mainframe Transaction Option process structure

The Single Execution Processes (SEPs) host a runtime CICS environment for a single transaction. The SEPs use process boundaries in Windows to provide the same fencing of memory resources as storage keys and protected instructions do in the System/390 architecture for mainframe CICS. Together, the SEPs form the traditional “CICS Region” for the transaction system and are managed as a group by the Directory Server (MFDS). The Communication Servers (MFCS) provide network input/output control to users. All components are linked through a shared memory space. The shared memory component provides terminal input/output areas (TIO, transaction COMM areas and other shared CICS structures. Although each application environment will vary, the scalability tests found that a ratio of 3.75 to 4.5 SEPs to physical CPUs in the partition is ideal for handling TPC-C workload. The scalability tests found that a larger number of SEPs increased Windows kernel execution time, primarily in increased context switching between processes. A lower SEP:CPU ratio resulted in starvation of CPUs and a lower attainable CPU utilization rate.

The numbers of users and the transaction rates scaled accordingly as the number of processors were increased during these tests. At its peak, 5160 user terminal sessions were simulated requiring a shared memory allocation of 1 GB and 2 MFCSs. Transaction environments that have different characteristics would likely be tuned differently. For example, long running transactions (“batch” jobs in the extreme) would require increased numbers of SEPs. Transactions that had a higher degree of interactions with others in the mix (increased contention for resources) would also require a larger SEP:CPU ratio since the SEPs are likely to be blocked at sometime during the transaction execution.

Scalability Results and Analysis

Note that the TPC benchmark rules prohibit publicly disclosing TPC performance figures that have not been independently audited. Therefore, we are required to withhold from this paper any data that may be used to derive our TPC metrics. However, we can disclose relative benchmark results from the different CPU configurations to convey the scalability of the system. Tests were performed by varying the simulated user count (see following table) or the number of CPUs in the configuration (see subsequent table) and observing its effect on cumulative CPU utilization. As with many TPC-C tests 90^th percentile response time requirements (under 5 seconds) led to the termination of the test before CPU was exhausted. Increases in response time were attributed to increased disk queue length and client customer networks.

The above table shows the effect of increasing workload in terms of concurrent users while maintaining a constant CPU configuration. As expected, the increased rate increased the transaction response time and CPU utilization consistent with the increased load.

Changes in the processor and disk subsystem hardware led to an increased MIPS rating. The earlier paper estimated 1347 mainframe MIPS equivalent for online transaction processing workloads. The 8-CPU system used in these tests equated to 1712 MIPS. The same versions of Windows, Enterprise Server with the Mainframe Transaction Option and IBM UDB were used here and in tests cited in the earlier paper. The hardware changes also allow significant increases in the number of concurrent users the system could sustain. In the earlier paper’s test the transaction response time requirements in the TPC-C spec limited the overall user count to 3600 for 1347 MIPS. Lower 90^th percentile response times were noted in these tests allowing the number of concurrent users to be increased to a high of 5160 (1712 MIPS).

A second round of tests concerned scalability of the alternative mainframe environment. We varied the CPU count (see following table) while keeping the number of concurrent users constant (except for the smallest configuration which could not sustain the higher count).

The above table indicates the effect of keeping the transaction rate relatively fixed while varying the number of CPUs in the configuration. In the two tables above, a 2-CPU case is included for comparison purposes and represents an off-the-shelf machine. Although capable of running the mainframe migration workload, it is evident these smaller systems use more CPU to process the same amount of transactions making them less efficient than “enterprise class” machines This is likely due to a limited I/O subsystem. The differences between the configurations show that a 16-CPU configuration is about 8x the speed of a 2-CPU configuration. The difference between 8 and 16-CPU configurations show a 1.9x performance advantage. Differences in the 12-CPU configuration may have led to the less than 50 percent improvement over the 8-CPU configuration. Further analysis of performance logs is required.

Conclusion

This paper has shown that continued improvements in CPU and disk subsystem design continue to improve upon the ability of Windows based systems to run mainframe workloads at high rates. An 8-CPU enterprise-class system is capable of 1715 MIPS of mainframe processing. Better than 80 percent of existing mainframe IT data centers are at or below this size. We feel comfortable predicting that any CICS COBOL transaction workload that can fit in mainframes of this size or smaller could be migrated to the architecture discussed here.

From the scalability tests we can conclude that Windows Server 2003 Datacenter Edition SP1 together with Micro Focus Enterprise Server with the Mainframe Transaction Option scales in a close-to-linear fashion for varying numbers of CPUs in the multiprocessor CPU configuration, having tested 12- and 16-CPU configurations. Evidence indicates that high-performance “enterprise-class” processing systems with adequate memory, high bandwidth to disk subsystems allow COBOL CICS transactions to execute effectively when moved off the mainframe.

Resources

• Mainframe migration: Performance of Enterprise Server, Mark Haynie, Micro Focus, 2004. [Mainframe Migration 2004]

• Characteristics of production database workloads and the TPC benchmarks, IBM Systems Journal, Vol. 40 No. 3, 2001. [Characteristics, 2001]

• TPC Benchmark C, Standard Specification, Revision 5.1, Transaction Processing Performance Council, December 2002, https://www.tpc.org. [TPC, 2002]

• The Mainframe Migration Alliance (MMA), a group of companies that are working together to help customers migrate workloads off the mainframe and onto the Microsoft platform. More information may be found at https://www.mainframemigration.org

• The Microsoft Partner Solution Center (MPSC) in building 25 on the Microsoft Redmond Campus in Redmond, WA. The MPSC provides complete hosting environments for product testing, evaluation, and scalability. Microsoft partners can leverage multiple vendors to support client engagements Proof of Concepts using cutting edging Microsoft Technologies.