What you find is not always what you fix—How other aspects than causes of accidents decide recommendations for remedial actions

[An update and repost of a paper that's amongst my favourites and most regularly referred to.]

By interviewing 22 accident investigators, they studied constraints that lead investigations away from the “ideal” assumption that “what-you-find-is-what-you-fix” (WYFIWYF).

That is, how does the same organisational context contributing to accidents constrain the investigation methods and understanding?

Results

Investigators believed that their investigations followed the ideal WYFIWYF; however data suggested that the analysis & design of actions are influenced by constraints.

Constraints, thus, practically influence how investigations take place, resulting in “what you find is not always what you fix” (p2137).

Constraints from the ideal included:

What you can fix depends on what you are able to understand: Relating to the experience of investigators (general & specialist), and knowledge of the task.

What you fix depends on what you can afford to investigate: Some investigators noted that resources were sometimes unavailable, & time was another major constraint limiting investigations.

What you fix depends on what you can find: Investigators noted difficulties in speaking to all of the relevant stakeholders, particularly outside of their own company. This includes not just the deceased, but also presumably companies that instantly dismiss employees.

What you fix depends on what you dare to find: Management wasn’t always happy about criticism, which interfered with examining management influences. This has obvious parallels to accidents where aspects of morality & failure to adhere to criticism about decisions being found in major accidents (e.g. Challenger).

What you are able to fix depends on what you find: This relates to the (sometimes problematic & arbitrary, in my view) stop-rules governing when to halt investigations. Various stop-rules were given by investigators, e.g. when “all the facts have been ticked-off”, when the event chain is (seemingly) described, when factors three steps up from event are detailed, or when about 5 fixes are identified.

What you know how to fix is what you fix: Relating to the point that some factors have known remedies, and investigators may just pick the ones that can easily be remedied. Authors note that while it may as first seem rational to fix what you know to fix, it may “lead to a hesitation to refrain from more complicated (but not yet known) possible strategies that may solve the problems” (p2135).

What you fix depends on what you find: In some domains, investigators thought that “new risks are not systematically considered” (nuclear), whereas in maritime it was believed that risks were considered but not systematically. Others recognised that some risks are hard to judge due to complexity and unforeseen consequences. E.g. Interventions to reduce risk may introduce new unforeseen risks. Recognised was that investigators may sometimes be naïve in suggesting actions that may increase complexity rather than reducing it.

What you fix depends on what you are going to fix: Investigations may identify fixes which were already being considered, and thus the report simply targets the preconceived fixes (or supports existing worldviews).

What you fix depends on what someone else decides: In some cases, some one else decides what recommendations to implement. This may be problematic since people at the source are potentially more familiar with the issues and uncertainties. However, being too familiar may increase the risk of them not seeing the bigger picture across domains.

What you fix depends on what is possible or who you are: Actions may not target the most appropriate things, but rather the easiest things (e.g. internally). Difficult fixes may also not be suggested.

What you fix depends on the cost-benefit balance: Although some investigators said cost wasn’t factored in, others recognised it was and influenced the types of actions. Sometimes, less expensive remedies were used while waiting for better ones.

What you fix depends on what is easy to understand and known to work: This includes solving known issues with existing means known to work - e.g. understandable, effective, tested, measurable, reliance on rules, even if these aren't necessarily the most targeted or suitable for the task.

Authors argue that although investigation methods are created in the hopes to be more thorough and complete, while generally being a good thing, it’s “unlikely to prevent most of the factors mentioned by the investigators from leading the investigation away from the ideal of [WYFIWYF]” (p2137).

That is, these effects are likely to be at play, to some degree, in all investigations irrespective of the method used.

Interestingly, no investigator mentioned poor investigation methods as a constraint.

Authors note the limited scope of investigation may be a source of bias in investigations to examine the bigger picture, where investigations become “limited to those causes that are currently seen as fixable by investigators” (p.2138).

It’s argued that accident investigation is not a rational process and neither is the choice & implementation of actions. Or, they say, perhaps it is actually rational, in a pragmatic way, to consider the possibilities & constraints of the situation and then selecting reasonable fixes.

In any case, they argue that “The bottom line must be a warning against believing in the rationality – but not the sensibility – of investigation and re-mediation” (p2138).

Authors: Jonas Lundberg, Carl Rollenhagen, Erik Hollnagel, 2010, Accident Analysis & Prevention