Metrology Monday! #96 A Discussion on Conformity Assessment, Decision Rules and Measurement Decision Risk – The Dobbert Method
I have shown you several sound decision rules that use guardbanding to control False Accept risk which are based on joint probability. To evaluate the measurement decision risk, we need to know the uncertainty of measurement, its level of confidence, the product specification and the End of Period Reliability (EOPR). The first three items are easy to obtain, but for the last one, we have made some assumptions about the estimate of EOPR. All the curves and information I have shared over the past few weeks have assumed a 95% EOPR. But in reality, this is only an estimate, an educated guess. For most applications, using this as an estimate will achieve acceptable results, but for those who want to provide a more accurate evaluation, we may want to dig a little deeper.
Back on post #90, I defined EOPR and how to calculate it. My question to our readers is, how often do you really evaluate EOPR for the products that you calibrate? To have an accurate value for EOPR, the larger the sample size, the better. Because of this some laboratories may calculate EOPR once a year. The reality is that many laboratories never calculate EOPR. I don’t say this to shame them, I only communicate this because it is a reality presently.
If you are a laboratory that doesn’t really know your EOPR, how can you evaluate False Accept and False Reject risks to ensure that they still meet your customers’ requirements? One of the most impressive metrologists that I have met set out to investigate and solve this dilemma. Michael Dobbert is a Metrologist for Keysight (formerly Agilent) Technologies. He started looking into this problem shortly after the release of ANSI/NCSL Z540.3-2006, which had a 2% or better requirement for False Accept risk. He published his paper in the 2008 NCSL International Workshop and Symposium. You can find his paper through Google fairly easily. If you have not read it, I would really recommend you do so, but I will at least cover some of the key points of his paper today.
Dobbert’s goal was to develop a guardbanding strategy that would always achieve no worse than 2% False Accept risk, not create excessively large False Reject risk, without ever having to calculate EOPR using joint probability. To do this, he computed False Accept risk curves for various TUR situations, for EOPR of 100% down to 0%. What he observed was that the maximum False Accept risk peaks is around 65% EOPR and it actually gets a bit smaller as it goes to zero.
The next step was truly inspired, and at the risk of using an over-used phrase, genius. He developed a guard band multiplier by setting the False accept risk to 2% in the joint probability formula and then solving for the multiplier.
Recommended by LinkedIn
This multiplier can be computed for any TUR by using equation 4, and then the guardband Acceptance limits can be computed by plugging the multiplier back into equation 3.
What is really fascinating to me is that if we build risk curves using the Dobbert Method (bright green lines below) and compare them to the risk curves for the RDS method evaluated at 95% EOPR (heavy blue lines below agree so well that they almost overlap!
Lastly, I will note that even at a 1.5:1 TUR, the False Reject risk never exceeds 20% regardless of EOPR, and at 2:1 it is only about 10%, so again the Dobbert Method really does a nice job of optimizing both False Accept and False reject risk and I think more organizations should be using it. #MetrologyMonday #FlukeMetrology
Superviseur qualité -Technologue en génie industriel
2wGreg Cenker thanks a lot !
I advocate the use of this algorithm extensively when teaching measurement uncertainty. It’s easy enough to implement, provides good results, and keeps PFR reasonable. The only time I recommend a deviation is when the customer wants a PFA of 1% or less, in which case (to match the Dobbert curve), the EOPR must be set to around 60%. Once this happens, nearly any PFA percentage can be calculated using the much more complex M1 approach. It’s really a great solution.
Author | Metrologist | President | Force & Torque Measurement Trainer.
3wAnother great read. The only real issue in the Dobbert method is that for high TURs above 4.6:1 is that it can and will push the acceptence zone past the specification zone. An easy fix if this bothers you is to use logic that says if TUR is above 4.6:1, use 4.6:1. Jeff, do you have any more to add on that?
"Came to Believe"
3wI find this research work even more impressive because people accomplish it without referring to any First Principles. As far as we can determine, none exist to help us with the core challenges. That isn't to say that we're making this up as we go, rather we are forced to feel our way along without any reference to intuition which often fails us when we employ statistics. Thanks again, Jeff!
“Metrology is the silent architect of progress, shaping precision into the foundation of innovation, trust and global harmonization.” - Nicol Best
3wVery interesting insights...How feasible is it for smaller laboratories with less computational capacity to adopt the Dobbert method effectively?