Use of “Control/Treatment Effectiveness Factor” in Cost Contingency Calculation
Pedram Danesh-Mand
President of Risk Engineering Society, Head of Project Controls & PMO at KPMG Australia
Most guidelines and best industry practices, including Risk Engineering Society (RES) Contingency Guideline, recommend that optimum contingency allowances for desired confidence level should be calculated by using the ‘residual risks’. But at the early phases of project development, e.g. Preliminary Business Case (PBC) or Final Business Case (FBC), how the ‘residual risks’ can be reasonably assessed from 'original risks'? Can ‘Control/Treatment Effectiveness Factor’ be used as a reasonable approach?
Last week, this question was a trigger between our Jacobs/Aquenta risk team and one of our key clients, a major utility investor/provider. This paper is a brief summary of hours interesting discussions and debates. These group discussions are fantastic way for sharing knowledge and experience between our team and our clients. Ongoing opportunities to challenge and validate all our assumptions, perceptions and perspectives by smart clients are priceless!
The topic of ‘Control Effectiveness’ or ‘Treatment Effectiveness’ in risk management has been a topic of debates for many years. These generally represent the overall effectiveness of key risk controls/treatment measures that may influence a particular risk or the overall risk exposure. From this definition, you will see that this factor not only represents the potential ability of measures to treat a risk, but also may impact the overall risk exposure directly driven by its consistency, completeness, KPI’s, accuracy, reliability and its timely application. How? Let’s discuss this through a simple example.
- A major lump sum (Design & Construct) tender submission by a contractor
- One of key risks: design may be delayed
- Few potential causes: not enough designers, low competency level for requirements, inefficient interface management internally, external stakeholder management and ineffective planning & controls
As you can see, depending on real cause of delay, there are, or will be, different controls or treatment actions which may affect the likelihood of the risk, i.e. preventive controls, or affect the consequences, i.e. mitigating controls. The wide range of actions may include getting more resources, specific personnel training, additional interface manager, additional design manager or additional planner. Forget about measuring effectiveness, selecting the right treatment action is the first challenge.
Due to these practical challenges, control effectiveness is often measured incompletely and inaccurately.
The figure below represents a typical good practice 3 phases risk assessment from ‘original risk’ to ‘residual risk’.
Few key points:
- Risk Exposure should be measured pre-control and then compared against Risk Capacity.
- Risk Profile should be measured pre-treatment and post-treatment and then compared against Risk Appetite.
- Residual Risks should be measured post-treatment and then be used for contingency calculation if still greater than Risk Appetite
However, I have seen some clients who use a range of ‘Treatment Effectiveness Factor’ as a reduction/adjustment factor on the Risk Profile at pre-treatment stage. The results will be then used to calculate cost contingency allowances. Is this a good practice?
Before I answer this, I would like to highlight my experience with C/TEF.
- First, in my view, C/TEF can’t be measured against individual control objectives. It can only be measured against the broader project or business objectives. How? This is for another long but interesting article!
- Controls can have unintended negative impacts on other objectives, so key unintended consequences should be clearly understood, assessed and accepted.
- In my experience, by providing an assessment on the level of residual risk profile (or status) for each key project objective, or risk category or business objective, we can implement a more reliable platform to assess the key areas of concerns. In other words, I believe we as risk and GRC managers should not only have a better understanding of the business objectives, but also a clear understanding of the risk appetite levels and risk exposure against risk capacity for each objective. It means, knowing the business in and out!
Getting back to question of using C/TEF in calculation of contingency:
In my view, this approach is reasonable mainly during project delivery phase in circumstances that there are a number of different treatment options available, making the selection of the best mitigation action difficult. To assist Project Manager in assessing the reasonable contingency, the C/TEF can be used subjectively to modify the required allowance. This method is more common being used by PM’s mainly during delivery phase to manage a considerable number of possible treatment options, hence a wide range of effectiveness.
However, at the early stages of project development, e.g. Final Business Case (FBC), the most recommended approach, as per a number of industry practices such as RES contingency guideline, the residual risks – i.e. post-treatment risk assessment results, recommended to be used for contingency calculations. The other reason for this recommendation is that during the development phase, all most likely treatment actions need to priced and moved to the Base Estimate. Hence, it’s better to quantify and assess the post-treatment residual risks thoroughly, hence there is no need for a reduction or adjustment factor.
Your feedback, comment or relevant experience are more than welcomed.
Delivering VALUE from Uncertainty
6yGood summary Pedram. I think that not all organizations truly understand what their risk appetite is, and even less ensure the "appetite" is fed in the estimate. I have seen both the "pre and post mitigation" numbers adjusted to lower the apparent risk profile, making it more palatable. I wholeheartedly agree that for any organization that has decided to engage in formal risk management, it must ensure that the discipline is engaged & involved on the business decision making process, not just a provider of inputs, reports, and recommendations.
Project Services | Project and Program Management | Contractor and Partner Oversight
6yGood discussion highlighting important topics. The fact that "risk controls" may have unintended negative impacts on other objectives is often overlooked. For similar reasons, the impacts must be understood during the development phase before moving any mitigation costs to the base estimate. Moving them to the base estimate requires they be ingrained into the project execution plan, scope of work, etc. through proper change management processes. Again, this often gets overlooked as being an "administrative task" already covered by staff costs. These can be very expensive mistakes.
Pedram Danesh-Mand - I agree with your conclusions. At an early stage, the team must keep focus on the broader project objectives, to price key treatment strategies into the baseline, and set aside a residual contingency. During project delivery, the project manager should be given flexibility to use contingency to action the treatments that give the best return as the project progresses.
Regional Manager at Ledcor
6yThank you for your clear discussions on this topic.
Senior Quantity Surveyor MAIQS, CQS
6yI agree with the recommendation of pricing and moving treatment actions to the Base Estimate during development phase, however this needs to be done with a number of other actions. The Base Estimate is the "Plan" and as such goes hand in hand with the Scope, the Schedule and the Risk Register. If the Base Estimate is changed, these other elements need to potentially change. Treatment Actions added and priced then become Controls and the corresponding risks in the register need to be amended to reflect that, by adding them to the Controls section, and indeed amending the post-control risk assessment. Furthermore Treatment Actions added to the Base Estimate may potentially increase activity durations and the projected completion date. My two cents worth, more of a reminder that any change to the Base Estimate at any stage needs to be controlled and managed, and the potential changes to the other key project planning building blocks need also be implemented.