Site Server - Risk Management Process for Product Development

December 1999 

Author: Dr. Joyce Statz
TeraQuest Metrics, Inc. 

1. What Do We Get From a Risk Management Process?

The primary deliverable from a team's risk management process is project team awareness of the things that could go wrong during the course of the project—the risks. Without such awareness, and without an effective process for managing the risks, the success of the project is left to chance.

The project team might record their current awareness as a list of risk items that they are managing. At any point, the team can examine that list of risks, review their mitigation actions, and build a risk profile chart that shows how well they have the risks under control.

2. Why Do We Care about Risk Management?

For many organizations, the most challenging and interesting work is that which deals with novel technologies, new applications of those technologies, and/or unusual environments for deployment of new or stable technologies. These are the opportunities that can really make a difference for the customer.

Commensurate with that opportunity is risk. Generally, the greater the opportunity for making a difference, the greater the risk that something will go wrong. To leverage the opportunities, we need to address the sources or the consequences of the risks, to ensure a successful project. We can't leave the results to chance.

We need to be concerned about risk management at two points for every project: when we initially consider a project being proposed for a customer, and as we run a project.

As we evaluate a prospective project, we want to understand any potential problems that could cause excessive cost, force the project to exceed the required time frame, compromise the functionality, or generate adverse publicity. In addition to the usual project management challenges, we want to identify the uncertain areas (the risks) that threaten the project. Identifying these risks allows us to know what exposure we are accepting (if we do nothing about them), or to take specific action against them. Our risk analysis gives us the understanding about where to add special talents to the team, or when to try multiple deployment paths simultaneously, or when to do extra levels of prototypes, or take some other action. It may also mean that we include our customer or a partner in the risk mitigation effort, since only they may be able to address or share some of the risks.

As we finish a project, we think through the lessons learned in running that project. Among those lessons are the risks which became problems, and why they occurred. The key to maximizing success on future projects is to learn what cues to watch for in the future, so that the risk is recognized before it becomes a problem. We capture those risk ideas and cues in risk factor charts, which are described below. When the factors apply to more than just one project team or one organization, we can share those in a common risk clearinghouse for overall organization learning.

3. Who Gets Involved in Risk Management?

Role

Type of Involvement

Project Manager [may be Program Manager, if working at that level]

· Drives the risk management process at the start of a project
· Participates in risk identification, mitigation, and tracking progress throughout the project
· Accepts or rejects the level of risk for the project

Project Team

· Performs the risk management process for this project

Risk Identification Team

· Provides input to the process for identifying risks
· Includes representatives of all affected groups involved in the project, as well as any others expected to have insight into risks for this project

Risk Mitigation Team

· Performs actions to reduce the exposure from this risk, focused on either or both of probability and consequence of the risk
· May be members of the project team, other affected groups, user, customer, management, and others, depending on the risk item

Process Improvement Team

· Maintain the organization's risk management process assets, incorporating lessons learned from projects

4. How Do We Perform Risk Management?

This table summarizes the risk management process that is detailed below.

Activity

Tasks Involved

Identify Project Risks

Select risk identification team
Select candidate risk categories to review
State specific project risks
Categorize risks

Analyze Risks

Estimate loss for each identified risk
Estimate probability of each identified risk
Calculate risk exposures (probability times loss)
Rank risks, based on exposures
Identify key risks to manage

Plan Risk Handling Actions

Determine strategies for avoidance or mitigation
Develop contingency plans
Document risk management approach
Develop current Top N risk list

Track and Control Risks

Track individual risk items
Update and report on risk status regularly
Monitor for new risks

Improve the Risk Management Process

Review risk management as part of post project analysis
Extend risk process and artifacts from lessons learned

Details about each activity follow: a description of the purpose, entry and exit criteria, and the sequence of tasks to be done for each activity. Tasks are shown along with the roles generally responsible and/or involved in those tasks. Examples at the end of the document show the content and relationship of some of the deliverables referred to in the activity descriptions.

4.1 Identify Project Risks

As the organization considers a prospective project, it evaluates potential risks to the opportunity, to be able to build a project plan that maximizes the probability of project success.

Purpose: Name and describe the specific risks faced in this project, to be able to analyze those risks and decide on an approach to handle them.

Entry Criteria: One of the following is true:

  • Customer has provided a request for proposal. 

  • Organization is proposing a project for the customer.

Roles

Tasks

Project Manager

Select risk identification team. Potential participants include:
· Project team
· Support groups (SQA, CM, test, documentation, training, etc.)
· Representatives from other elements of the program
· People from other related projects
· Partner representative
· Customer representative
· User representative

Risk Identification Team

Identify sets of risk factors that may apply. Examples include:
· General risk factor chart (or one tailored to an organization)
· Specific risk factor chart for this type of project
· Lessons learned on previous projects in this organization
Identify which risk factors are relevant to the project, and rate their potential for indicating risk to the project (high, medium, low) or 1 to 5.

Risk Identification Team

For each high factor, identify the specific risk(s) to the project, citing the condition that could arise and the consequence to the project if it does arise. (See examples at end of document.)

Risk Identification Team

Organize the specific risks into categories that support analysis of impact and developing countermeasures. (These vary by project.)

Project Manager

Determine which of the risks to analyze further.

Exit Criteria: List of risk items to be analyzed.

4.2. Analyze Risks

The risks identified are analyzed to establish the overall project exposure to risk and to determine which risk items are the most important ones to address. In rare cases, the overall project risk exposure will be so high that the opportunities represented by the project really cannot be attained at a reasonable expense. In most cases, though, the project opportunity will be maximized by attacking the most significant of the risk items.

While the initial risk analysis deals with those risks identified early in the project, more analysis may be needed as the project proceeds. In cases where a new risk is identified, that new risk is analyzed and its exposure compared to that of other risks already being handled. That new risk may or may not be addressed with a mitigation action, depending on the cost of that action and the ranking of this new risk against others already being handled.

Purpose: Determine the projected risk exposure for each identified risk item.

Entry Criteria: Identification process is complete and there is a team which can review the risks to estimate their impact.

Roles

Tasks

Project Manager

Select process to be used for analyzing impact of selected risks. Alternatives include:
· Binary analysis (two values on dimensions of each of probability and impact)
· Tri-level analysis (three values on dimensions of each of probability and impact)
· Top N analysis (probability values on one dimension and losses by economic or other loss levels on the impact dimension) [description that follows this uses Top N]

Risk Identification Team

Review each risk item and estimate:
· Probability of occurrence of this risk item
· Loss if the risk occurs
Calculate the risk exposure for each risk item.
Rank the identified risks in order of exposure.
Review ranked results and ensure team agreement with the ordering.

Project Manager

Examine the full list of risks and determine the overall level of risk for this project. (Different methods of summing exposures or counting number of risks above a certain exposure might be used by different practices.)
Compare the overall level of risk against what this organization is prepared (or authorized) to take on. (This determination will make use of a database of information about the level of risk encountered in various projects, to be built over time. Initially, judgments will need to be made, based on estimates that reflect past performance.)
If the project is to proceed, identify those risks which merit mitigation efforts.

Exit Criteria: [For initial assessment] A decision is made about whether or not to proceed with the project.

An acceptable level of risk for this project is agreed upon by the project team, project manager, and others involved in the approval processes for the project.

If the project proceeds, a list of key risks is agreed upon by the Project Manager and project team as those to be handled.

4.3. Plan Risk Handling Actions

Risks may be handled a number of different ways. Alternatives include:

  • Accept the risk, with no investment of effort or cost. This is appropriate when the cost of mitigating exceeds the exposure, and the exposure is acceptable. 

  • Transfer the risk to someone else, or agree to share the risk. If the customer or a partner is better able to handle the risk, this is probably the most effective approach to eliminate the risk for the project as a whole. 

  • Fund and staff efforts to reduce the probability that the risk will become a problem. Such mitigation tasks might include providing additional staff to help develop the product, getting special training for members of the team, or following a dual development path for the whole project. 

  • Fund and staff efforts to reduce the loss associated with the risk should it become a problem. Examples might include keeping a backup LAN operational during the deployment of a new network, or providing free training to users who would otherwise not be trained. 

For significant risks that cannot be mitigated, and for key risks for which countermeasures are unreliable, contingency plans may need to be established and then executed if the risk becomes a problem.

Purpose: Establish an affordable set of actions to minimize the exposure from the key risks identified and ensure project success.

Entry Criteria: List of key risks has been prioritized.

Roles

Tasks

Project Team and Project Manager

For each key risk, identify an approach to handle the risk, and estimate the effort or cost required for that action.
For risks which require them, identify contingency plans.

Project Manager

Incorporate the risk handling actions into the project plan.
Document contingency plans and their anticipated cost and effort, should they be needed. Establish agreement with management (or other funding source) about how to decide when and if to authorize use of the contingency plan.

Exit Criteria: All key risks have been addressed with actions and/or contingency plans, which are cost -justified against the benefits to the project.

4.4. Track and Control Risks

Throughout the project, the project team tracks progress in mitigating the risks, to ensure that:

  • Actions which should reduce the probability of occurrence are effective 

  • Actions which should reduce the loss associated with the risk are effective 

  • Risks for which there is no possible mitigation action have not reached a trigger point for the contingency plan. 

In addition, the team watches for additional risks that need to be addressed, as well as changes in impact or probability to previously identified risks.

Purpose: Ensure that mitigation actions are keeping the risks under control, and monitor indicators to know when to invoke contingency plans.

Entry Criteria: Risk handling actions are staffed and funded. Contingency plans are defined where appropriate.

Roles

Tasks

Project Manager

Regularly review and update the status for each key risk, to ensure risks are under control.
Update and publish the current Top N Risk chart.
For any risk out of control, revise the mitigation action or get approval to proceed with the associated contingency plan.
Prepare a risk status report for use in project reviews.

Project Team, others working with the team

Be alert to other potential risks and communicate them to the project team.
Participate in regular review and updating of the current risk list.

Exit Criteria: Risks to the project are at or below the level agreed to as acceptable for this project.

4.5. Improve the Risk Management Process

There are several natural points at which to evaluate the effectiveness of the risk management process: when a long project completes a phase, or when a project is complete (or terminated).

At each of these points, the project team should conduct a post-project analysis to evaluate the success of the project, including a review of how well the risks were managed. Based on the results of the examination, changes may be made to the tables used for identifying risks or to the various processes used to analyze, plan for, and handle the risks.

Purpose: Review the effectiveness of the risk management process, and make changes to accommodate learning from the project.

Entry Criteria: Project has reached its conclusion or has completed a phase.

Roles

Tasks

Project Manager

Organize the post-project analysis session, including results of risk management
· Risks that were detected initially and successfully handled
· Risks that were detected during the project, but not identified at the start
· Problems that arose during the project, but were not detected as risks at any point

Project Team

Identify lessons for future risk management
· Countermeasures that were effective
· Contingency actions that were successful
· Changes to the ineffective countermeasures
· Identify changes to risk factors for risk identification in future
· New factors to include in the appropriate risk factor table
· Factors that can be removed from the table
· Changes in the cues provided in the chart for high, medium, and low risk

Process Improvement Team

Incorporate results of analysis into the risk factor chart and risk management process

Exit Criteria: Risk management process elements updated

5. Work Products

The following work products are created or used by this process.

Work Product

Used or Created

Risk Factor Tables

Used, updated

Top N Risk List

Created, updated

Detailed Risk Tracking Chart

Created, updated

Risk Status Report

Created, updated

6. Tailoring Guidelines

The Risk Management Process will be used in different ways during the start up of a project and during the ongoing execution of a project. The items below summarize the key differences in those two uses.

Activity

Project Start up

Project Execution

Identify Risks

Detailed; covers all categories of potential risk to project effort and to the organization

Team is alert to specific single risks as they may arise

Analyze Risks

Focused on opportunity represented by project and level of risk to overall project; decide go/no go

Focused on what additional resources need to be applied to mitigate a new risk; review the opportunity provided by the project against the cost of mitigating this risk (as well as others)

Plan Risk Handling Actions

Builds the initial set of plans and commitments for mitigating risk and for contingency plans

Determines where the action for this new risk ranks against others already underway; may or may not mitigate, depending on cost/benefit analysis

Track and Control Risks

 

Monitor throughout project, based on currently known set

Improve the Risk Management Process

At project completion, explicitly consider initial risk management activities

Focus on the handling of ongoing risk identification, analysis, and mitigation

7. Examples of Risk Process Items

Following are examples of risk management work products created while using the Risk Management process. Information about these items should be made available in detail in the training about risk management.

7.1. Factor Tables

The full collection of risk factors for all projects can be segmented into categories; some categories apply to all projects, others to certain types of projects. For each factor, there might be cues as to what constitutes high, medium, or low risk within that factor for a project. For example, the category of Customer/User factors might have the following factors and indicators:

Customer/User Factors 

Risk Factor

Low Risk Cues

Medium Risk Cues

High Risk Cues

User Involvement

Users highly involved with project team, provide significant input

Users play minor roles, moderate impact on system

Minimal or no user involvement; little user input

User Experience

Users highly experienced in similar projects; have specific ideas of how needs can be met

Users have experience with similar projects and have needs in mind

Users have no previous experience with similar projects; unsure of how needs can be met

User Training Needs

User training needs considered; training in progress or plan in place

User training needs considered; no training yet or training plan is in development

Requirements not identified or not addressed

User Justification

User justification complete, accurate, sound

User justification provided, complete with some questions about applicability

No satisfactory justification for system

Since the types of projects done by an organization generally carry the same types of risks, the organization can have a risk factor table with the categories and factors that apply to all of its projects. In addition, it can have tailored tables with just the factors that affect specific types of projects, such as those using a new technology. Each project, then, starts by examining the relevant factor tables, to see which of those factors suggest risks to this project.

7.2. Overall Project Risk

A risk team may examine the risks for a particular project against organization history, to see how this project compares. If this project has more risk than those which have been successful in the past, the project may merit special risk mitigation actions, as well as careful review of progress. If the project is about the same level of risk as others which were successful, the team should follow similar practices as in the past.

The team might build up an overall project risk table, showing the level of risk they see in each potential category of risk which their organization monitors. The chart below shows an example of how they might review the risk on this project (Mean Rating) against acceptable levels set in the past.

For illustration purposes only, suppose that the project risk categories included only the 5 categories in the chart below. Suppose that for the current project, a mean rating was developed as a mean of the exposure ratings of all factors rated for each category. Suppose that the Acceptable Mean Rating is the highest level at which the organization has been successful in the past with this category. For this project, then, it appears that the team can be successful, since the ratings in each category are below the Acceptable Mean Rating.

 

Risk Category

Mean Rating

Acceptable Mean Rating

1

Customer/User Factors

4.25

4.5

2

Service provider Skills and Abilities

2.5

3.5

3

Account Team Factors

2.0

3.0

4

Customer Environment

3.5

4.0

5

Customer Organization Management

3.0

4.0

 

Totals

15.25

19

7.3. Risk Statements

A risk to a project needs to be understood in very specific terms, so that an appropriate action can be formulated to address the risks. The risk statement has two parts:

  • Condition—an expression of what could go wrong, the cause of a potential problem 

  • Consequence—a description of the impact that potential problem could have on the project 

For the factors represented in the Customer/User category above, some possible risk statements for projects might be:

  • Because there is only one person from the customer organization available to work with us on the project, the project team is making decisions about what to include in the project specification. This may lead to solutions which are not valid for the customer's evolving business. 

  • Because this project was initiated without a strong business case by the CIO who has since left the organization, there is little assurance the team will be able to carry the project beyond the initial feasibility study. 

In each case, there are actions that can be taken to minimize the risk exposure by attacking the condition, and there are actions which could address the consequence. In these cases, each requires collaboration with the customer.

7.4. Top N Risks

As part of risk analysis and planning, the identification team estimates the probability and loss for each risk, and it selects a way to handle the risk. Summary information about the risks can be shown in a Top N Risks chart.

Here the team estimates a high loss for the first risk (the project will be potentially quite ineffective), and a smaller loss for the second and third (the project will take more time, but the project personnel will eventually be available). The probabilities (Prob) that these risks will become problems are high for all three. The risk exposure (Exp - product of probability and loss) clearly shows that the first two represent larger threats to the project's success than the third risk does. The risk chart is generally maintained in order by exposure.

ID

Risk Item (Condition and Consequence)

Prob

Loss

Exp

Risk Handling Approach

Who

Date

1

Lack of customer personnel involvement leads to invalid conclusions on needs of business

.7

8

5.6

Discuss results with customer management team to get sanity check

Requirements team

2/15

2

Lack of organization strategy experts now lengthens the schedule and entails an opportunity cost.

.8

5

4.0

Seek strategy experts from the home office and other field offices

Project manager

1/15

3

Reports provided are inconsistent, requiring more interviews of personnel by the organization service provider

.8

3

2.4

Ask for supporting documents from those who build the architecture.

Architecture team lead

1/15