Metrology Monday! #91 A Discussion on Conformity Assessment, Decision Rules and Measurement Decision Risk – Birth of the 4:1 Ratio

The 4:1 ratio is something that has been cemented into calibration programs all around the United States and has also leaked out to the rest of the world.  I find it to be generally misunderstood and often misquoted.  Today I am going to take you back to the invention of the 4:1 Ratio.

The person who came up with this idea originally was Jerry Hayes, while working for the U.S. Navy.  If there was a Mount Rushmore for people associated with Calibration Quality Standard development, Jerry would surely be on it.  He fundamentally conceived of most of the requirements for Calibration Quality Systems that we use today.  Jerry was a friend of mine, and I kept in contact with him, sometimes seeing him at the Measurement Science Conference, and sometimes just emailing or calling him after he fully retired.  This information is based on interviews and conversations that I had with him a couple of years prior to his passing.

World War II had several technological breakthroughs never seen before by humankind.  This war saw the invention of the Atom bomb, the rocket propelled missile, and the jet engine.  These were devices that began their existence as a scientific experiment, where engineers and scientists were trying to make the first one.  By the end of World War II, the U.S. government was requiring them to be mass-produced.  Oftentimes, the U.S. government would require, for the purpose of secrecy, that contractors only build part of these systems, and the government would assemble them upon receipt into an end product.  Because of the challenging tolerances for both dimensional and electronic components, when the government received them, they either did not fit together or did not perform properly. The reason for this was calibration errors.

Jerry discovered this while working for the U.S. Naval Ordinance Laboratory in the mid-1950’s.  He worked in the Missile Evaluation Department Product Quality Division.  The problem that they were facing was that the missiles the Navy was receiving from contractors were not reliable, nor were their associated test reports.  He was tasked with problem-solving to determine why this was.  He produced his results in Technical Memorandum 24 October 1955 No. 63-106 “Factors Affecting Measurement Reliability.”  He concluded that calibration quality issues were a root cause of missile reliability.  He wrote, “The systems of selection and measurement controls as established by the calibration and compatibility programs have been set forth in this paper as a means of pinpointing and containing measurement problems.”  After he presented this information to his superiors, they directed him to go out and build a full system to control these calibration issues. 

In those days, computers were not widely available, so most mathematical work had to be done by hand and with slide-rules (if you don’t know what that is, look it up!).  Originally, he set out to achieve a goal of 10:1 in his calibration program based on MIL-STD-120, a military standard for gage inspection (dimensional standards).  However, he found that 10:1 was not achievable with microwave electronics associated with missile test sets.  He drew inspiration from two papers, Alan Eagle’s “A Method of Handling Errors in Testing and Measuring” and Frank Grubbs’ and Helen Coon’s paper “On Setting Test Limits Relative to Specification Limits.” Both were published in Industrial Quality and Control, March 1954.

The target for False Accept and False Reject established by leadership (with some coaxing/influence from Jerry) were both set to a goal of 1%.  Both the estimate for measurement uncertainty and End of Period Reliability (EOPR) were set to 95%, approximately 2-sigma.  When Jerry did the math to solve this problem, he observed that if the ratio of the missile test limits to the instrumentation specification limits were 4:1 that the False Accept risk was computed to be just under 1%, and the False Reject risk was just over 1%.  Encouraged with this result, he brought this information back to his management.  They loved the idea because it was easy to compute, and it met their risk goal requirements.

At this point it is very important to note that the 4:1 ratio was based on Joint Probability, not Conditional Probability.  It is also important to understand that this goal was based on False Accept and False Reject requirements, and that a 4:1 ratio under the right circumstances can produce this.  A 4:1 ratio will not produce 1% False Accept risk if the EOPR is not close to 95%.   A 4:1 Ratio can have much less than 1% False Accept Risk if the measurement uncertainty or EOPR is higher than 2-sigma.  For those that do not like the 4:1 Ratio or Joint Probability, they must understand and concede that almost everything we do in calibration today is predicated on these ideas.  The 4:1 ratio is a decision rule that allows you to have confidence that you have 1% probability of False Accept risk for any measurement from -100% to +100% of the test limits. #MetrologyMonday #FlukeMetrology