A pilot test verifies that the solution meets the expectations and specifications of the customer by testing the entire solution in a production environment. Typically, a pilot test is a deployment in a subset of the live production environment. This subset can be a particular group of users, an entire department, or a group of servers in a data center. The pilot testing process helps identify areas where users have trouble understanding, learning, and using the solution. It also gives Release Management the opportunity to identify issues that can prevent successful deployment of the solution.
During the pilot test, the team collects and evaluates pilot data, such as user feedback. Once the team has collected enough data, the team must choose one of the following strategies:
Stagger forward. Deploy a new release candidate to the pilot group.
Roll back. Execute the rollback plan to restore the pilot group to its previous configuration state. Try the pilot again later with a more stable release candidate.
Suspend. Suspend pilot testing.
Patch and continue. Deploy patches to fix the existing solution.
Deploy. Proceed to deployment of the solution.
Table 5. Activities and Considerations for Conducting a Pilot Test
Pilot the solution and collect feedback
Is the solution viable in a production environment?
Are all of the solution’s components ready for deployment?
Have the project team and customers agreed on pilot success criteria?
Does the project team have a mechanism in place for collecting feedback?
Which sites and groups have the project team chosen for the pilot? Are there one or more user profiles the team is targeting for the pilot test?
Are targeted users prepared for the pilot?
Has the project team documented a transition plan in the pilot plan?
How will the project team evaluate the results of the pilot test?
What risks are associated with pilot testing the solution in production?
Master project plan, including:
Pilot-ready release candidate, including:
Pilot review document
Release notes updated
Do not begin a pilot test until the project team, customers, and users all agree on the criteria for a successful result.
Pilot the entire deployment process and not just user interaction.
Communicate to users the length, purpose, and progress of the pilot.
Ensure that support is trained and prepared to support pilot users.