DP Incidents Nov/24

Introduction:  It’s time to look at some of the DP related incidents and reports over the last month.  These will be broken into “Rumors”, things that actually happened but with limited information or sources I can’t share, ”Non-DP”, things that can happen on DP but didn’t, and “Reports”, documented findings.  It’s important to realize that we are not smarter, luckier, or better than the people that these happened too, and that we need to put ourselves in the place of the participants, so we can take each threat seriously.   We need to remember that each incident could have been us, and protect against these expensive lessons.

“Rumors” – actual DP events but limited info

 Real & Fake Trials:  There are two types of annual trials.  One is based on careful analysis and testing of the vessel, and its purpose is to ensure that the vessel is still capable of DP operation.  It shows that the equipment is working and systems capable of supporting safe operation.  It comes with some risk, as the tester may find problems and will have to investigate in order to verify continued safe DP operation.  The other type of trials follows a rigid procedure provided by the vessel.  The third party may witness the trials, but cannot adapt them to the vessel and its situation.  This lowers the risk of something being found, makes the trials shorter, and allows the vessel to get back to work.  The first is proper and right, but the second may not be an annual trial.  It looks like an annual trial – a procedure was followed, an independent witness provided, and it has the logo and name of a third party provider on the report.  But it wasn’t an annual trial verifying safe DP capacity, just a report verifying that a test program was carried out.  These are not the same thing and should not be accepted as the same thing by vessel clients.  A real annual trials looks for problems and proves capacity, while a cover trial performs set tests.  Real trials usually find problems and cover trials usually hope not too.  This has been a problem for a long time.  Cover trials use the reputation and name of the third party to distract from the limited test and survey.  The third party is trading short term money for degraded long term reputation, and some consultancies have gone under because of this.  I’m not giving any recent examples of bad practice or the resulting incidents, but we have seen them for decades and need to do better.  MTS has free TechOp O-02 for evaluating annual trials procedures, and IMCA M190 was updated last year.  The guidance in both are helpful in detecting problems.  Anything worth doing is worth doing right.

DGPS Addiction:  I have heard about a number of DGPS problems and of course the continued jamming and spoofing problems in the Middle East.  Nothing is new there.  DGPSs were never redundant on their own, and operators need to find ways to protect themselves.

UPS Problems:  In a shocking case of false advertising, they appear to be interruptible after all.  Problems have included lack of dependability, lack of endurance, and hidden crossovers in the supply or load.  Proper grounding of the UPS distribution is vital, load noise can be disruptive and may need monitored to detect problems, and UPS endurance testing needs regularly performed with the worst case failure load rather than what is convenient.  Some designers can’t help putting in automatic changeover switches (ACOS) to crossover power because they think it will improve redundancy when they are ruining it.  Manual crossovers need opened, crossover ACOSs need locked out at the crossover supply, and protections need regularly tested.  Nothing new here, but a rash of problems came home to roost on some vessels.  Don’t forget that your UPSs only accept certain frequency ranges and may need surge suppressors on their supply or load side as their protections are limited.

Video:  I saw a good video reminding people of our purpose.  This is the diver's view of a drive off.  I don't know when it happened, and it is probably an old incident.  It was posted Oct 18/24 and I missed it at the time, but it's a good reminder of what we want to prevent.  Found here.

“Non-DP” - didn’t happen on DP

HMI:  The picture comes from a UK Marine Accident Investigation Report (pg17).  Bad layouts aren’t usually as blatant as this, so it’s a good reminder to think through the controls on your vessel, look for traps, and protect against accidents.  They could have made the controls clearer by adding arrows, as shown in the picture, but would have been better making the movements consistent in the first place.  A small cruise ship punctured the hull above the water line as it was berthing.  This was caused by the previous reason and a more universal one – they were distracted by troubleshooting an equipment failure as they came into the berth.  Troubleshooting and operating at the same time is a bad idea – pick one.  Distraction during operation is dangerous.  Don’t let solving a problem take away situational awareness.

Update:  The HMNZS Manawanui inquiry has reported and some of the initial rumors reported in the Oct/24 DP incident article were wrong. The vessel didn't blackout and drift onto the shoal. It accelerated into it. Instead of using autotrack in DP, the naval crew were using a mixture of autopilot and manual controls. They got confused and thought they had taken manual control and were wondering why the thrusters didn't respond, when they autopiloted into the shoal at 10 knots. Blackout came after abandonment. Three unnamed crew members will be disciplined. 

More Problem Solving: There are groups of people who would rather do everything manually, rather than depend on a machine. It's a viewpoint that I partially understand, as sometimes the machine isn't appropriate and manual skills need maintained with use. But they were semi-automated, because they were sometimes using the autopilot and sometimes making manual turns. It was that last failed turn, when they thought they were in manual, but were actually in autopilot, that got them. Sometimes people go into a befuddled solving problem mode, when they should be in safe operation mode. It's not the first time and won't be the last.

Routine is a problem. Doing something automatically and unthinkingly is a danger. Being present, purposeful, and self-critical is key to preventing and catching problems. People usually are and do, but we need to remind ourselves that we could all make this kind of mistake, if not deliberate in our decisions and actions. Detecting the skipped step is hard, unless you are willing to reject the assumption that you just did it right, like you always do. It's even harder when it's a group. Independent skepticism or mechanical fault finding procedures can be useful.

“Reports” - DP findings from public sources

Study:  Tugfan Sahin announced his study on DP incidents had been accepted for publishing and I asked him to share a summary.  He was kinder than that.  He shared his paper in the comments to his announcement (link no longer works) and he gave permission for me to summarize here.  Most of my readers have more than a passing interest in safe DP operation, so that was great.  His analysis was based on the IMCA 2004-2021 DP incident reports.  He looked at 1352 reported incidents, had to throw out 600 of them for poor data, and then applied statistics to identify significant findings (association rule mining to discover robust associations).  This is more in depth analysis then normally found in the IMCA annual DP incident reports (that we are grateful for). 

Findings:  They started with the typical analysis you find in an IMCA report and then continued by filtering the data and finding strong association rules, such as feedback failures causing thruster failures, thruster trips causing thruster failures, DGPS degradation causing position reference faults (it’s the DGPS addiction, you see), software failures causing computer failures, and the environment causing drift off.  None of these are surprises, but they are a base for the analysis.  The analysis of drift off found that human factors were strongly associated with loss of position whether incorrect commands, accidentally going into standby, or not reacting to sudden changes.  The human factor had the strongest relationship with drift off, followed by environment, while loss of redundancy was normally a technical fault, such as loss of power, auxiliary failure, and control system failure.  These were always in combination with other factors.  Looking at individual factors, the strongest association was squalls, accidentally going into standby, improper wiring, waves, and sudden changes in current causing drift offs, and loss of an OS, thruster, or UPS causing loss of redundancy.  Looking for strong associations between operation type showed shuttle tankers associated with drift off due to human error, diving associated with power drift offs, pipelay human factor drift offs, drill losing redundancy due to power or thruster faults, OSVs losing redundancy due to power fault, control fault, or thruster trip, and ROV losing redundancy due to control faults.  Both power and thruster faults were strongly associated with loss of position.

A Grain of Salt:  So, how can this be used?  Are dive ships really more likely to blackout and drift off, or are they more likely to report it?  The incidents are self-reported and probably less than half a percent of the actual incidents.  There is a self-selection reporting bias where the most professional ships provide the least incriminating data, and the least professional ships have mysteriously perfect records (and little chance to learn).  Despite the analysis being based on statistically suspect self-reported data, it found things that sound right.  The emphasis on the human factor in the findings was interesting and should be a focus of improved operation.  I find a lot of failures come from improper setup and failure to recognize and make safe on loss of redundancy in my think through of the incidents.  Faults like a thruster trip shouldn't be significant if the vessel is operated redundantly and within its capability.  Of course, that is a problem, just like DGPS vulnerability caused by excessive dependence.  They do emphasize buttonitis and lack of knowledge.  They didn't think through the distorting effect of self-reporting and should have wondered why different ships using the same technology have very different failure modes.

Conclusion:  We can get mad and claim that we would have never made these mistakes, but we are no better than the people that these happened too.  The old hands will note that most of the incidents are familiar faults that we have encountered again and again.  I encourage everyone to become familiar with the old faults, so we can make less of them.  Share what you learn from your own incidents with others by submitting incident reports. Captured lessons are lessons learned. I hope you have found something interesting and useful that you can apply to make your own work safer.