Defining the Testing Methodology
A major part of your test plan is describing the strategy for testing. When planning your methodology, consider:
Where will the testing take place?
Who will perform the tests?
How will you communicate with and involve participants?
How will you schedule the testing?
How will you manage application problems?
Outsourcing is one option for application testing. To determine if you will use this option, consider the following:
Do you have staff available for testing?
Does your staff have the appropriate level of expertise?
What are the internal costs compared to the outsourcing costs?
What is your time frame? Can the testing get done faster if you outsource it?
What are your security requirements? Would you need to provide confidential data to an external organization?
When you test internally, select experienced testers. If your organization has a group of application testers, it is recommended that you use them. If you do not have such a group or they are unavailable, look for ways to use a variety of resources to achieve the best results in a reasonable amount of time. For example, you can use a few experienced testers to develop a battery of test cases, which they can train others to run. Alternatively, you might have the experienced testers perform a core set of tests and then coordinate with business units to have their experts come to the lab to perform the functions they use in their work.
Devise a process for scheduling test days and communicating with the testers. For example, you might set up a Web site on your intranet where anyone can view test dates, status reports, contact names, and other relevant documents.
Establish a procedure for managing test results. Describe roles and responsibilities, including the following:
Who enters problem reports in the incident tracking system?
How are problems prioritized, assigned, and resolved?
Who tracks the resolution of problems and retesting of applications?
How do testers enter test results in the test tracking and reporting system?
Case Study 1: Testing Festivals
A large high-technology organization asked its developers to test applications for compatibility with Windows 2000. The test manager worked with other managers to get cooperation from their teams. Because the test manager reported to the CIO and had total support for the program, it was easier for her to get active participation. She scheduled testing sessions in the lab and sent notifications about the sessions to the developers. The lab was set up with preconfigured computers, ready for the testers to install their applications. Testers could install their applications from CDs or from the network. To make the events fun, the test manager provided food and beverages, thus earning the name "test fests" for the sessions.
Case Study 2: Preview Program
A major manufacturing company developed a preview program for testing Windows 2000 Professional. They used this program to test client-based applications. This organization first verified that the protocol stack to be used on the Windows 2000 client computers was compatible with its Windows NT 4.0 production environment. Then it deployed Windows 2000 Professional at restricted locations on the production network to create a place for testing applications.
The project team set up a Web site with information about the program. Users filled out an application form on the Web site to apply for participation in the preview program. To limit the number of participants and to ensure thorough testing, the project leader and the employee's manager reviewed and approved applications. The program, which was restricted to 50–100 participants, provided a wide range of testers for testing applications. Testers posted their problem reports at the Web site.