False Electrical Redundancy

Introduction: Someone was complaining to me yesterday about having to fight with shipyards and vendors, because they don’t understand dynamic positioning (DP) redundancy and do things that ruin good electrical designs. It’s not just them, I’ve seen coast guard and class comments ruin perfectly good designs, and crew “improve” good designs to destruction in the field. It’s repeating the obvious, and already well documented in the DP guidelines, but people like overcomplicating the simple, so let’s clear out some conceptual dead wood.

Exhibit 1 – Death by Automatic Changeover Switch (ACOS): “Don’t the circuits in the title picture look great? Sure, it’s a nightmare to run all the wires and maintain the extra equipment, but the joy of a job done well, and really putting your mark on the design, makes it all worthwhile. It’s a monument to the ages, a control power supply system that always supplies power to the DP vital loads. And the picture is just an overview – all the loads are fed like this. This system will never lose power! Allow me to illustrate with the picture below:”

Exhibit 1 Ideal: “When one of the supplies fails, the ACOS dependent on that supply automatically transfers to the other supply. The port UPS output failed low and the affected ACOSs automatically switched to ensure continued power. If the supplies were big enough, the system could lose one, then two UPSs, and still be redundant. It could lose three UPSs and still keep running. Breakers will protect the supply against any conceivable load fault, so it’s a solid design.”

Exhibit 1 Reality: Everything needs maintained to work and, in the offshore, maintenance is kept to the bare minimum or below. Most designers don’t conceive of these operating limitations and design systems that won’t work well in low maintenance environments. This is an example. When properly maintained and regularly exercised, the ACOS won’t get stuck or be slow to transfer, but this is a DP vessel where the rule is “Don’t play with working systems on DP”, and the vessel is usually on DP or going between work sites. Maintenance is overhead and downtime, so if the relays and breakers get worked on every five years, then that owner is a hero. There are DP vessels that have gone decades without maintenance on important protections. I know this from the accident investigations. The original assumption of an acceptable transfer speed may have also been wrong. Transfer and reset or restart is not a win. Finally, the expectation that an unmaintained, overcurrent trip, molded circuit case breaker will protect against all significant faults, shows a breathtaking lack of imagination or experience.

Exhibit 1 Debunked: Allowing every load to be automatically fed from any source, allows any major load fault to be connected to all sources, if it trips the first one, transfers to the second one and trips it, transfers to the third one and trips it, and finally transfers to the last one and kills it. That’s what the picture shows above – a port load fault took out all equipment in all four redundancy groups. “But won’t the breakers prevent that?” You mean those uncalibrated and unserviced things that only protect against simple faults? Sure, they improve the odds. They certainly don’t block all faults and, in DP, we believe in Murphy’s Law and look to the Worse Case Failure (WCF). We don’t say that the breaker should work. Instead, we notice that the breaker improves the odds, but doesn’t protect against some faults. The breakers might not trip (I’ve tested lots) and earth faults, harmonic noise, uneven loads, and spike loading might take out the supply without tripping the breaker. Faults aren’t always continuous and active faults can bypass protections. The breaker might trip, if it was given a nice simple current overload, but the world is complex and control loads are active. Loads aren’t the only problem, as faults in the ACOS, or the wiring between, can throw faults onto two or all supplies. If the relay coil pops, both supplies might be effected. Supplies that are good for their current load may not be good for extra load, even if faults were blocked and transfer reliable. If you have read the last few articles, the IMCA DP incident reports, and the DP guidelines, or worked in the field, then you know I’m not making these up.

Exhibit 2 – Death by a Thousand Diodes: Some designers realize that mechanical switches can be unreliable and go for the solid state solution. Diodes are comparatively instant and reliable. This isn’t useful for AC control power, but is a standard DC power solution. The picture shows the DC equivalent of the ACOS setup. The diodes automatically select the highest voltage supply, so all four supplies can be combined together. A common bus is not used to feed them all, as a common bus fault could take out all power, and is equivalent to a single supply.

Worse: While the DC equivalent is faster and more reliable than its AC equivalent, it is far worse. A faulty AC supply could only disrupt the equipment that it was attached to, but the diodes always select the highest voltage, so any supply that fails high affects all loads. If the AC comes through the rectifier, if the output is biased, if the output is high, if there is spike noise on one supply, or if one supply is unstable and reacts to noise load or EMI, then all the loads are affected. Diodes might be more reliable than mechanical switches, but they still fail, and their failure may not be apparent or simple. They also need regularly tested or monitored. An open diode or connection may prevent a load fault from going through to one supply but also means the expected backup supply is not available, while a shorted diode directly connects one supply to the load bus and may act as a load on a higher voltage supply. The diodes and breakers do not protect against all important faults and usually aren’t maintained, so we are back to the problems found in the ACOS system, with the multiple supplies interconnected to faults and reacting to them and each other. These failure modes can be complex.

Exhibit 3 – DC Improved: This is the modern improvement on the old Exhibit 2 design. The DC to DC power supplies provide electrical isolation between the supply and load, limit overvoltages, prevent overloads, and provide supply boost. The redundancy modules provide diode combination of the supplies and provide diode health monitoring. These features are almost standard design - although only two supplies are normally being combined, and one of them may be an AC supply, via an AC to DC power supply. It is certainly more reliable and fault resistant, but it is not perfect. They are worn out by heavy use (electrical and physical environment), the internal capacitors have a ten year lifespan, the diode monitoring is often not hooked up to indication or alarm, and they are not usually tested. This means that they will go bad as some point, and no one will know until it fails to prevent a fault. It’s time to consider reliability. This might be good enough for DP2, where failure only needs to be less likely than fire or flood. Even then, these modules should be replaced every ten years or tested yearly (OneStep Power Solutions had a paper on this at the 2022 MTS DP conference and I have an article). If we are honest, this may not be good enough for DP3, as even a protected cross-connection cannot be considered as safe as two open breakers (dual isolation).

DP3 Separation: Remember the picture above? The most perfect electrical protection system is equivalent to no protection at all, if it is in the same space. Designers often put all the protection in the same box. We can fill that space with water, burn it down with fire, or fill the box with the plasma of an electric arc, and the protection still needs to work. That doesn’t work if the protection is all in the same box or space. Some reviewers are happy with two breakers in a row on each supply outside the space, but as previously discussed, that is an uncalibrated, unmaintained protection that is not designed to resolve complex faults.  Two simple breakers aren’t enough. A DC to DC power supply isn’t enough. External protection needs to be intelligently designed to cover all possible failure modes, rigorously maintained, regularly tested, and external to the fire/flood space to count. You can guess how few DP3 vessels meet that standard. We give lip service to separation and wonder why DP3 vessels fail just as often as DP2 ones.

Exhibit 4 – KISS: After all the complicated power drawings, the above system might be a disappointment. That’s a problem, because it meets DP2 and DP3 requirements. Each redundancy group can live or die on its own, and the lack of cross-connections prevents electrical faults from spreading to other redundancy groups. We don’t want to lose thrusters, but DP vessels are designed to have redundant thrusters, so we would rather lose some than lose everything or too much. If we are still redundant after the loss of one group, we can even keep working. This is an ideal and a simplification, but it is a basic principle that needs to be absorbed. We don’t want to lose a redundancy group, but we would rather lose one redundancy group than two or more. All of the previous complications put multiple redundancy groups as risk. When designing or making improvements, this ideal should be the conceptual baseline. Does the proposed change add more benefits than risk and fault severity? Are these trade-offs understood? They usually aren’t and focus is often solely on the imagined benefit. This simple split philosophy is a reality check that needs used to challenge ill-formed concepts. Independent redundancy groups is the first design criteria.

Exhibit 5 – KISS Improved: As shown above, we don’t need to cross feed power to have redundant supplies. While we only want to lose one redundancy group, we want to avoid it and to restrict the losses to as little of the redundancy group as possible. The picture doesn’t shown the type of power combination in the loads, as the drawing is a general example, but it should be arranged to limit a UPS or load fault to its source. This won’t always work, but with no crossover power, the worst power failure will be limited to one redundancy group. Again, this is partially an ideal, but it is an extremely informative one that should be used to inform electric design, installation, and maintenance. Fault limitation is the second design criteria.

Conclusion: MTS and IMCA already tell us to avoid these crossover connections. Hopefully, this article will inspire someone to keep things simple, and support the DP redundancy concept, rather than making an electrical masterpiece that cannot be maintained and creates operating risk. Now we just need to convince owners not to buy snake oil, and vendors not to sell it. Experienced reviewers know to expect trouble when they see use of ACOS or diodes in DP2, or any use of crossover power in DP3. Designers, installers, maintainers, and operators need to start thinking the same way.  Keep It Split and Simple.