Risk Rules

When it comes to risk management, hope is not a strategy ... all single-point estimates are wrong ... and communication is everything. Understanding these principles and two others is the only way to turn risk management theory into meaningful practice.

Risk management is essential for the success of any significant project. Information about key project cost, performance, and schedule attributes is often unknown until the project is underway. Risks that can be identified early in the project and that might impact the project later are often termed “known unknowns.” These risks can be mitigated, reduced or retired with a risk management process. For those risks that are beyond the vision of the project team, a properly implemented risk management process can rapidly quantify the risk’s impact and provide sound plans for mitigating its effect.

 Risk management is concerned with the outcomes of a future event whose impact is unknown, and dealing with this uncertainty. Outcomes are categorized as favorable or unfavorable. Risk management is the art and science of planning, assessing, handling, and monitoring future events to ensure favorable outcomes. A good risk management process is proactive and fundamentally different than reactive issue management or problem-solving. Risk management can be better understood by five key concepts:

1.  Hope is not a strategy. Hoping that something positive happens will not lead to success. Preparing for success is the basis of success.
2. All single-point estimates are wrong. Single point estimates of cost, schedule, and technical performance are no better than 50/50 guesses in the absence of knowledge about the variances of the underlying distribution.
3. If you do not integrate Cost, Schedule, and Technical Performance, you are driving while looking in the rearview mirror. The effort to produce the product or service and the resulting value cannot be made without making these connections.
4. If you do not apply a model for risk management, you are driving in the dark with the headlights off. Risk management is not an ad hoc process that you can make up as you go. A formal foundation is needed, one that has worked in high-risk domains.
5. Risk Communication is everything. Identifying risks without communicating them is a waste of time.

Risk management is an important skill that can be applied to a wide variety of projects. In an era of downsizing, consolidation, shrinking budgets, increasing technological sophistication, and shorter development times, risk management provides valuable insights to help key project personnel plan for risks. It alerts them to potential risk issues, which can then be analyzed to develop, implement and monitor plans to address risks before they adversely affect project cost, performance, and schedule.

Hope is Not a Strategy

Hoping that the project will proceed as planned is not a strategy for success. The same can be said for project managers who constantly seek ways to eliminate or control risk, variance, and uncertainty. Managing in the presence of risk, variance, and uncertainty is the key to success. Some projects have few uncertainties — only the complexity of tasks and relationships is important — but most projects are characterized by several types of uncertainty. Although each uncertainty type is distinct, a single project may encounter some combination of four types:

1.  Variation describes many small influences yields a range of values on a particular activity. Attempting to control these variances outside their natural boundaries is a waste of time.
2. Foreseen Uncertainty refers to identifiable and understood influences that the team can’t be sure will occur. There needs to be a mitigation plan for them.
3. Unforeseen Uncertainty can’t be identified during project planning. When it occurs, a new plan is needed.
4. Chaos appears in the presence of “unknown unknowns”

 Plans are strategies for the successful completion of a project, which are different from schedules. Schedules show how the project will be executed. Plans show what accomplishments must be performed, and the success criteria for these accomplishments along the way to completion. The plan describes the increasing maturity of the project through maturity assessment points. The unit of measure for this maturity must be meaningful to the stakeholders — something that can be connected to the investment they have made in the project.

When we speak of “hope,” it lays the foundation for failure. Hope really means “success is possible but not probable.” When we speak the word “plan” it does not assure success, but success is a probable outcome. It is the definition of the probability of success P(s), that is the foundation of the Plan. Having a Plan A, Plan B, and possibly a Plan C exposes risk, assigns mitigations, and measures the probability of success.

 The idea of a Plan as a Strategy is critical to making changes in the behavior of the project teams that can lead to “risk-adjusted project management.” Without a Plan, the schedule is simply a list of activities to be performed. The reason for their performance may be understood, but it is unlikely these activities fit in any cohesive Strategy. Strategies have goals, critical success factors, and key performance indicators.

No Single Point Estimates

How long will this take? How much is it going to cost? What is the confidence in those two numbers? These are three questions that must be answered for the project team to have a credible discussion with the stakeholders about success. Deciding what accuracy is needed to provide a credible answer is a starting point. But that does not address the question, “How can that accuracy be obtained.”

There are many checklists for estimating cost and schedule, with simple guidance on how to build estimates. Most of this advice is wrong in a fundamental way. The numbers produced by the estimating process do not have their variance defined in any statistically sound manner. By statistically sound it means that the underlying probability distributions are known. If they are no known, then some form of estimating taking this unknown into account must be used.

 The Project Management Institute advises producing three estimates — optimistic, most likely, and pessimistic. But these can be fraught with error. How are these numbers arrived at? Are they based on best judgment? Historical data? What is the variance on the variance of this distribution — the second standard deviation? In the absence of this information, they are of little use in estimating risk.

 The use of point estimates for duration and cost is the first approach in an organization low on the project management maturity scale. Understanding that cost and durations are actually random variables drawn from an underlying distribution of possible value is the starting point for managing in the presence of uncertainty.

 In probability theory, every random variable is attributed to a probability distribution. The probability distribution associated with cost or duration describes the variance of these random variables. A common distribution of probabilistic estimates for cost and schedule is the Triangle Distribution.

 The Triangle Distribution can be used as a subjective description of a population for which there is only limited sample data, and especially where the relationship between variables is known but data is scarce. It is based on the knowledge of the minimum and maximum and a “best guess” of the modal value (the Most Likely).

Using the Triangle Distribution for cost and duration, a Monte Carlo simulation of the network of activities and their costs can be performed. In technical terms, Monte Carlo methods numerically transform and integrate the posterior quantitative risk assessment into a confidence interval. The result is a “confidence” model for the cost and completion times for the project based on the upper and lower bounds of each distribution assigned to the duration and cost.

 Integrating Cost, Schedule, and Technical Performance

In many project management methods, cost, schedule, and quality are described as an “Iron Triangle.” Change one and the other two must change. 

This is too narrow a view of what's happening on a project. It’s the Technical Performance Measurement that replaces Quality. Quality is one Technical Performance measure.

 Cost and Schedule are obvious elements of the project. Technical Performance Measures (TPM) describe the status of the technical achievement of the project at any point in time. The planned technical achievement is part of the Performance Measurement Baseline (PMB).

The Technical Performance Measurement System (TPMS) uses the techniques of risk analysis and probability to provide project managers with the early warnings needed to avoid unplanned costs and slippage in the schedule. Systems engineering uses technical performance measurements to balance cost, schedule, and performance throughout the project life cycle. 

 Connecting Cost, Schedule, and Technical Performance Measures closes the loop on how well a project is achieving its technical performance requirements while maintaining its cost and schedule goals. IEEE 1220, EIA 632, and "A Guide to the Project Management Body of Knowledge“ provide guidance for TPM planning and measurement and for integrating TPM with cost and schedule performance measures (Earned Value).

Technical performance measurements compare actual versus planned technical development and design. They report the degree to which system requirements are met in terms of performance, cost, schedule, and progress in implementing risk retirement. Technical Performance Measures are traceable to user-defined capabilities. Integrating these three attributes produces a Performance Measurement Baseline that:

 … is a plan driven by product quality requirements rather than work or effort requirements.

… focuses on technical maturity and quality, in addition to cost and schedule.

… focuses on progress toward meeting the success criteria of technical reviews.

… enables insightful variance analysis.

… ensures a lean and cost-effective approach to project planning and controls.

… enables scalable scope and complexity depending on risk.

… integrates risk management activities with the performance measurement baseline.

… integrates risk management outcomes into the Estimate at Completion.

 The Cost and Schedule “measures” are straightforward in most cases. The measures of Technical Performance involve measures of Effectiveness and Performance. Measures of Effectiveness (MOE) are the operational mission success factor defined by the customer. These are:

1.       Stated from the customer's point of view

2.       Focused on the most critical mission performance needs

3.       Independent of any particular solution

4.       Actual measures at the end of development

Measures of Performance (MOP) characterize physical or functional attributes relating to the system operation:

5.       Supplier’s point of view

6.       Measured under specified testing or operational conditions

7.       Assesses delivered solution performance against critical system level specified requirements

8.       Risk indicators that are monitored progressively 

Well-Defined Process

Using an ad hoc risk management process is itself risky. The first place to start to look for risk management processes is where managing risk is mandatory – aerospace, defense, and mission-critical projects and projects. These also include ERP and Enterprise IT projects.

 Technical performance is a concept absent from the traditional approaches to risk management. Yet it is the primary driver of risk in many technology-intensive projects. Cost growth and schedule slippage often occur when unrealistically high levels of performance are required and little flexibility is provided to degrade performance during the course of the project. Quality is often a cause rather than an impact to the project and can generally be broken down into Cost, Performance, and Schedule components. The framework shown below provides guidance for:

• Risk management policy
• Risk management structure
• Risk Management Process Model
• Organizational and behavioral considerations for implementing risk management
• The performance dimension of the consequence of occurrence
• The performance dimension of Monte Carlo simulation modeling
• A structured approach for developing a risk handling strategy

Risk Communication

To be effective the activities of risk management must properly communicate risk to all the participants. Risk is usually a term to be avoided in normal business. Being in the risk management business is not desirable in most businesses – except insurance. It is common to “avoid” the discussion of risk.

 Communicating risk is the first step in managing risk. Listing the risks and making them public is necessary but far from sufficient. Risk communication is the basis of risk mitigation and retirement. It serves no purpose to have a risk management plan and defined mitigations in the absence of risk communication. The Risk Management Plan must address:

 Executive summary – a short summary of the project and the risks associated with the activities of the project. Each risk needs an ordinal rank, a planned mitigation is the risk is active (a risk approved by the Risk Board), and the mitigations are shown in the schedule with associated costs.

 Project description – a detailed description of the project and the risk associated with each of the deliverables.

Risk reduction activities by phase – using some formal risk management process that connects risk, mitigation, and the IMS. The efforts for mitigation need to be in the schedule.

 Risk management methodology – using the NASA Risk-Informed Decision-Making process is a good start. This approach is proven and approved by high-risk, high-reward projects. The steps in the processes are not optional and should be executed for ALL risk processes.

 In order to communicate risk, clear and concise language is needed. English is not the best choice. Ambiguity and interpretation are two issues. Communicating in mathematical terms is also a problem, since the symbols and units of measure may be confusing. Figure 5 is from the Active Risk Manager tool that connects risk management with the scheduling system. ARM is a proprietary risk management system, but illustrates how risk is retired over time in accordance with a plan. This approach shows explicitly when each risk will be “bought down” or “retired” during the project execution. The Risk Registry and the Integrated Master Schedule must be connected in some way. Without this connection, there is no Risk Management process that can be used to forecast impacts on cost or schedule.

At each project maturity point, current risks, the planned retirements of these risks, and the impact of the project must be visible in the schedule. With these connections, project managers can then answer the questions:

• What happens if this risk is not mitigated?
• What effort is needed to retire this risk before a specific point in time?
• If this risk becomes an issue, what is Plan B? How much will Plan B cost? What is the impact of Plan B on the deliverables?
• What cost and schedule reserve is needed to cover all the currently active risks?

 Wrap Up

Once cost, schedule, and technical performance are integrated into the Performance Measurement Baseline, risk management can be applied to all three elements. With these connections in place, the project management team can say with confidence, “We are doing risk management on this project.”

 The final reminder is to make sure all five elements of risk management are present. Leaving one out not only reduces the effectiveness of the risk management process but increases the risk to the project. Project risk management is a Practice. The theory of Project Risk Management is important, but the Practice is how project risk gets managed.