How Does Selective Reporting Distort Understanding of Workplace Injuries

This open access paper from Kevin Geddert, Drew Rae and Sid Dekker is a very interesting read. It applied a new method for studying under-reporting of injuries in company data and for also revealing systematic biases.

I feel like I haven’t done a good job explaining this, so I recommend just reading the paper.

Company injury data (from an Australian energy company) was matched to the workplace injury insurance provider data using a bespoke method called one-to-one injury matching. As noted by the authors, this method allows two points of view:

1) the company/Incident Management System (IMS) view, which reflects measurement behaviours from industry, and

2) the view of incidents from a government insuring body.

The authors nicely summarise some key issues with relying on reported injuries for measuring safety performance: (p1)

·        Lack of statistical significance of changes in injuries due to the low numbers of injuries.

·        Injury rates driven by lower severity and more common injuries, which may be misleading about HiPos.

·        Number and nature of reported injuries being more sensitive to changes in reporting and classification of the injuries than to real differences in safety performance.

A key goal was to explore whether “LTIFR data are far more indicative of changes in claiming behaviour and claims management than of changes in OHS performance” (p1).

Results

Using the one-to-one injury method, it was found that only around 19% of insurance claims were disposed as recordable injuries in the IMS. All of these were said to be accepted by the insurer with some level of medical treatment or lost time.

They identify two ways that injuries were incorrectly classified in the studied company: (p12)

·        Being classified as “first aid only” injuries despite the need for more serious medical treatment.

·        Being classified as “not work related” despite the insurer decision that the injury was caused by work.

Also of interest is that only 17% of high severity injuries and 33% of very high severity injuries were considered recordable injuries. Furthermore, demonstrated was that if using a “recordable” classification scheme is to help identify the most severe injuries in this category, then this may not be happening in practice where “an injury marked as recordable is just as likely to be less than Medium severity as it is to be above Medium severity (3 of 10 in each case Related”, and of “the 12 High and Very High severity injuries, 3 were categorised as “recordable”, 1 was categorised as “first aid”, and 8 were categorised as not work related” (p7).

Although high and very high severity injuries are slightly more likely to be disposed as recordable compared with the less severe injuries, several cases involving significant medical costs and lost time were still recorded as “not work related” or “first aid only” rather than as recordables. Further, the total cost of “not work related” injuries seem to exceed the total cost of recordable injuries.

For what types of injuries are classified as recordable – while overall 19% of injuries in this company are classified as recordable, 30% of hand injuries but only 8% of back injuries were recordable. Expanding here, it’s said that several other body parts “all also associated with exertion or chronic injury, have a high likelihood of being incorrectly disposed” (p13).

In discussing the findings it’s said that “the definitions of “work-related”, “first-aid” and “medical treatment within industry have significantly departed from the insurer definitions” (p12) and that these narrower definitions are evident in the patterns of injury types classified as recordable.

Consistency across research, including this study, suggests a more systematic problem exists across industry for distorted injury reporting and this likely isn’t related to companies “hiding injuries” but more likely wider pressures to reduce the number of recordable injuries for tendering, insurance premiums and the like.

Interestingly, they state “Our interpretation is that the pressure stems from the use of recordable injuries as a performance measure, both within companies and along supply chains” (p12).

Also of interest is that the risk assessment ratings generated in the injury classification had little to no recognisable connection to the actual injury impacts. There was in contrast a close link between the risk rating and whether an injury was disposed as recordable.

It’s said that selection biases in reporting “has the potential to create an echo-chamber in which some of the most serious threats to safety are misunderstood” (p13). This is compounded by the point that a consistent measurable definition of actual incident severity doesn’t exist and thus, the authors aptly state that “industry has an altered perception of risk that is more aligned with “recordable injury” definitions than actual injury outcomes” (p14).

This problem also impacts other ways of learning from incidents, such that moderate or lower severity injuries may be treated “as either statistically equivalent to a fatality, or statistically irrelevant, with no in-between” (p14) and pressuring the classification and justification of moderate severity injuries as non-events, like first aid or not work related events.

Finally, the authors nicely argue that until additional large-scale research across multiple companies is conducted, “the default assumption for any company should be that reportable injuries are not representative of the severity and nature of actual injuries” (p14).

Link in comments.

Authors: Kevin Geddert, Sidney Dekker, Andrew Rae, 2021, Safety