Microsoft Commerce Server 2000: Site Testing

This chapter provides a general methodology for testing software, and then provides more specific information about how to test your Microsoft Commerce Server 2000 site. Testing is essential for any Commerce Server implementation to ensure software reliability and system performance and capacity.

Testing Methodology

Testing is a key part of the software development process. This section describes the types of testing you should do and the documents you must create to define your testing process.

Types of Tests

The following table lists the five types of tests you should use in your testing process.

Tests can be manual, scripted, or automated. The types of testing you do can vary, but should always include BVT, functionality, and ad hoc tests. Performance testing is especially important for an e-commerce application. The following table lists the two types of test methods that you can apply to each type of test.

Types of Test Documents

There are three types of documents to create before you begin testing your site:

• Test plan: Strategy and schedule for testing your Commerce Server site.

• Test cases: Detailed, step-by-step descriptions of each test.

• Test error reports: Reports that describe errors encountered during testing.

Test Plan

Developing a good test plan is key to successful testing. In fact, it can help define the application. To create an effective test plan, you must first understand your company's strategic vision for the site, and then define your test strategy accordingly.

A test plan includes the strategy and scheduling information for the entire project—everything except descriptions of the test cases. The test plan lists the features that need testing, the required resources, the risks you need to consider, and the testing schedule. The results of your test-planning process should be a stable specification, an accurate schedule, and a test plan containing an effective test strategy.

The test plan should reiterate the goals of the site and divide the site into testing areas. In addition, the plan should include inter-group procedure information, such as how bugs will be tracked and resolved, how test releases will be handled, how shared components will be tested, and criteria for determining whether a build is accepted or rejected for testing. Finally, the test plan must identify which testing tasks will be automated and which testing tasks will be done manually.

Test plans should describe in detail everything that the test team, the program management team, and the development team need to know about the testing to be done.

The following table lists the sections of a test plan.

Test Cases

A test case is a detailed, step-by-step description of a test. The following table lists four basic types of test cases.

You can organize test cases in a variety of ways, depending on the size, type, and complexity of the site you are testing. For example, you can organize test cases by type of test, by site feature, or by a combination of the two.

You can organize test cases by test type, such as the following:

• BVT tests

• Functional tests

• Integration tests

• Stress tests

Alternatively, you can organize test cases by product feature, such as the following:

• Primary-level features or functions

• Secondary-level features or functions

• Tertiary-level features or functions

Depending on the complexity of the site, you might organize test cases using a combination of methods, such as the following:

• Primary level:

  • BVT tests

  • Functional tests

  • Integration tests

• Secondary level:

  • BVT tests

  • Functional tests

  • Integration tests

• Tertiary level:

  • BVT tests

  • Functional tests

  • Integration tests

  • Stress tests

The following table lists the sections of a test case.

Test Error Reports

A test error report is a report that describes errors encountered during testing. An error, or bug, occurs when the site doesn't perform the way it was designed to perform. For example, if you attempt to add an item to your shopping basket, but the application doesn't add the item, an error has occurred. The following table lists the information that a test error report should contain.

Testing a Commerce Server Site

Before you begin testing, you must set up your test environment.

Important Create a separate test environment in which to do your testing. Do not install new content or software in your production environment before it has been thoroughly tested.

For best results, you should create a test environment that exactly duplicates your production environment. Testing each tier of the architecture separately, as shown in the following list, also helps to reduce the complexity of the testing task. You must also test the security of each tier. The following is a sample tiered architecture:

• User interface layer (top tier)

  • Visual appearance and usability

  • Links

  • Browser compatibility

• Business logic layer (middle tier)

  • Software performance testing (business logic, tax, shipping calculations)

  • Server load

  • Stress

  • Gateways

• Database layer (bottom tier)

  • Search results

  • Query response time

  • Data integrity

  • Data validity

  • Recovery

User Interface Layer (Top Tier)

To be sure that your site is attractive and user-friendly, you should test the following elements in the user interface layer (top tier):

• Visual appearance and usability

• Links

• Browser compatibility

Visual Appearance and Usability

The visual appearance and usability of a Web site, including its catalog pages, is important because a visually pleasing and easy-to-use site enhances the customer experience and encourages repeat visits. The following table lists the elements that you should test in the user interface layer.

The following list describes some other tests you should perform. Test to be sure that:

• Each toolbar and menu item can be used to navigate correctly using either the mouse or the keyboard.

• It's possible to navigate correctly through all pages using either the mouse or the keyboard.

• You have used proper format masks. For example, all drop-down lists should be sorted alphabetically. The date entry should also be properly formatted.

• Colors, fonts, and font widths are standard for the field prompts and displayed text.

• Vertical scroll bars or horizontal scroll bars do not appear unless needed.

• The window can be resized.

• All of the text displayed in each window is spelled correctly, including the window caption, status bar options, field prompts, pop-up text, and error messages.

• All character or alphanumeric fields are left-aligned (depending on the language).

• All numeric fields are right-aligned.

• Defaults are displayed, if there are any.

• All windows have the same look and feel.

• The tab order is from top left to bottom right, avoiding read-only or unavailable fields in the tab sequence.

• The cursor is positioned on the first input field when a window is opened.

• Each control behaves correctly, including push buttons, radio buttons, list boxes, and so on.

• The command buttons appear dimmed when not in use.

Links

Hyperlinks in Web sites can be broken, missing, or improperly assigned, making it impossible for a site visitor to navigate to the appropriate Web page. Make sure that all links work properly. You can use third-party software to search for broken and incorrect links; however, you must search for missing links manually.

The following table describes the types of links you should test.

The following Web sites offer online testing for broken and incorrect links at a nominal fee:

• http://netmechanic.com/ 

• http://www.linkalarm.com/index.html 

Browser Compatibility

Text, graphics, or colors can appear differently, depending on the browser used to look at them. To ensure that your site appears the way you want it to look on all major browsers, you should use development software that is compatible with most of the popular browsers, such as Microsoft Internet Explorer, Netscape Navigator, America Online (AOL), and Microsoft WebTV.

The following table compares the compatibility of browsers with common Web site components.

Business Logic Layer (Middle Tier)

Middle-tier testing primarily tests software performance and load. You test software performance to ensure that the software performs in accordance with operational specifications for response time, processing costs, storage use, and printed output, regardless of the volume of traffic on your site.

If your servers are overloaded during peak traffic volumes, you might have to invest in additional Web servers to prevent downtime and to enable you to offload traffic from an overloaded server during peak times. The ability of a Web server to handle a heavy load at peak hours depends on network speed and the server's processing power, memory, and storage space.

You should gather data on software performance during:

• Current and expected normal transaction volumes

• Current and expected peak transaction volumes

• Minimal transaction volumes

You should test all components fully, including facilities and equipment, and make sure that your local area networks (LANs) and wide area networks (WANs) are performing satisfactorily.

The following table lists the elements you should test in the middle tier.

Database Layer (Bottom Tier)

The database for a Commerce Server site typically contains data for catalogs, shopping baskets, user profiles, and orders. For this reason, it's extremely important to test the database layer to ensure the integrity of your data. Database testing is an ongoing process because no database is static. When you create a database, you should also create a copy and store it either on the same computer or on another computer. Run your tests on the copy of the database, not the original. When your tests are successful, you can install the necessary changes in the original database.

The database does not have to be, and usually isn't, located on the same server that hosts your Web site's storefront. The database server should be separated from the Web server by a firewall, which adds complexity to the testing processes.

The following table lists the elements that you should test in the database layer.

The following list describes some tests you should do to ensure data integrity. Make sure that:

• Data creation, modification, and deletion in data tables work as specified.

• Sets of radio buttons represent a fixed set of values. Test what happens when a blank value is retrieved from the database.

• Each value is saved fully when a particular set of data is saved to the database. Strings should not be truncated and numbers should not be rounded off.

• Default values are saved in the database if a user doesn't enter other values.

• New data is compatible with existing data, hardware, versions of the operating system, and interfaces with other software.

Security

You must test security on all parts of your site because ensuring the security of transactions over the Internet is important to gaining customer confidence. Gaining and keeping the confidence of online customers is extremely important to the success of your site.

The main technique for testing security is to attempt to violate built-in security controls. You must make sure that system protection mechanisms can prevent unauthorized access. You should then attempt to overwhelm the system by continuous requests, thereby denying service to others. You might purposely cause system errors during recovery or browse through unsecured data to find any possible keys to system entry.

The following table lists types of security breaches that you should test for.

For more information about best practices for setting up security on your site, see Designing Secure Web-Based Applications for Microsoft Windows 2000 by Michael Howard (published by Microsoft Press). There are two primary areas of concern in e-commerce security: network security and payment transaction security.

Networks

Unauthorized users, or hackers, can wreak havoc on your Web site by accessing confidential information or by damaging data. To prevent unauthorized access, you can configure the network operating system and the firewall to manage network security.

You must configure the network operating system to allow only authentic users to access the network. You must also install and configure firewalls to ensure that data transfer is restricted to only one point on the network. This effectively prevents hackers from accessing data.

For example, if hackers access an unsecured FTP port (such as Port 25) on a Web server, they could use it as an entry point to the network and access data on the server. The hackers might also be able to access any computer connected to this server. You should design your security tests to reveal vulnerable areas and highlight the network settings that need to be reconfigured to enhance security.

You can use third-party programs to test for the following:

• User rights

• Removable disk locations

• Strong password policies

• Use of logon scripts and password expiration dates

• Passwords stored in clear text or encrypted form

For more information about testing network security, see http://www.intrusion.com/ .

Payment Transactions

When customers purchase goods and services over the Internet, they can be apprehensive about providing their credit card information. To make your site secure for your customers, you must ensure that the following two conditions are met:

• Credit card information is transmitted and stored securely.

• Strong encryption software is used to store credit card information, and only limited, authorized access to this information is allowed .

For more information about secure electronic transactions, see the following third-party sites:

• http://www.verisign.com/ 

• http://www.spyrus.com/ 

• http://www.cybercash.com/ 

• http://www.checkfree.com/