It’s Time to Upgrade IJS to IDP
With dynamic positioning operators (DPOs) increasingly less capable of maintaining position, after a dynamic positioning (DP) control system failure, with the independent joystick system (IJS) or manual controls, it is probably time to upgrade IJS systems to independent DP systems (IDP). I’m hardly the first person to think or say this and have opposed this in the past, but I’ve come around to that position. DP control systems aren’t redundant and the redundancy comes from the DPO intervention but an IDP (independent DP control system) seems to be a more reliable solution than the IJS. Let’s have a look at what is required and why.
Credit: Let’s start by giving credit where it is due. Suman Muddusetti presented this idea years ago but I was so anchored to the idea of independently minded and conscientious DPOs ensuring that they were ready that I could not accept it at the time. I didn’t think owners would accept the expense and just thought that operators needed to let DPOs practice more. I was wrong. Russell Hodge kick started me thinking about this again, when he advocated for IDP last week. Russell and Suman are right. Of course, some owners were far ahead of the curve and there have been vessels equipped with DP capable IJS for decades. These insightful operators are a minority but they probably shouldn’t be.
Why IJS Exists: There is a psychological tendency called anchoring, which is often portrayed as a fault, but is a mixed blessing and curse. Anchoring is the tendency to hold on to an accepted idea until the preponderance of evidence disproves it. Whether anchoring is a virtue, which allows consistency despite noise, or a fault, which causes good evidence to be rejected, really depends on the evidence threshold. Redundancy and redundant systems are terms and phrases used a lot in the DP industry and this causes people to believe that DP control systems actually are redundant. They aren’t. Duplicated hardware, with the same design and manufacturing, same software, same network, and same environment, has common weaknesses and these common weaknesses negate redundancy. The smart money knows this and that is why we have IJS to take over after the DP control system fails. The DP control system isn’t redundant or smart enough to know that it has failed, so we have operators to select independent control after the automatic system and its linked backups have failed. This is no surprise, as the more carefully designed and regulated aircraft and nuclear industries also need operators to keep their systems safe. What is surprising is that modern DP manufacturers’ FMEAs (failure mode and effect analysis) tend to downplay this as extremely unlikely, or not even bother listing the possibility. It is still taken seriously by the rules that demand IJSs and by operators who see far more failures then the optimistic engineering/sales projections would predict. Redundancy actually depends on the DPO reliability taking IJS control and maintaining position when the non-redundant DP control system fails.
Problem: The DPO “reliably” taking and maintaining position is the problem. With manufacturers downplaying the possibility and some modern industry guidance, such as IMCA, downplaying the need for corrective operator action, the importance of the DPO reliably taking IJS control to maintain position is downplayed. As systems become more reliable, DPOs need to do this less and less, and are less capable of doing it in an emergency, unless they regularly drill on the vessel specific task. With some important industry advice not supporting regular IJS drills and operators under constant cost and time pressure, DPOs are not getting those drills. When I started in DP, almost any DPO could maintain position with the IJS or manual levers. In fact, many could reliably outperform the IJS and if the manual controls were independent then that was fine. Now, most crews that I encounter cannot maintain position with either the manual controls or the IJS. Some of them have to find the IJS or even get it out of storage for the annual trials. Most struggle to operate the IJS and can’t maintain position. Most DPOs don’t setup their equipment, so they have a reference to position against, after a failure. For example, some screen displays are normally setup to look pretty and show the outline of the vessel rather than zoomed in to display the movement of the control point within warning and alarm watch circles that take up most of the screen. Single pixel displays are hard to use as references. Of course, after the DP control system fails, there may be no DP screens, so the DPOs need to set the position set point on the DGPS screens and scale the screen properly before the DP system fails, so they can use them asr a reference after the DP system fails. DGPS isn’t always the primary position reference but this thinking applies to other primary references. Almost no one does this unless reminded and some DPOs don’t know how. All these problems could be improved with increased training and better guidance, but the underlying industry pressures that cause these problems still remain. It’s not right to blame the DPOs for doing what they have been told. These industry distortions mean that we need to make it easier for DPOs to reliably perform this task, and that’s where IDP comes in.
Solution: The basic idea behind the IDP is to provide a technical aid to the DPO to improve the reliability of emergency backup position keeping. The industry is no longer interested in giving DPOs enough vessel specific time to play with the equipment and develop enough vessel specific familiarity to reliably operate it in an emergency. And while owners hate spending money, they prefer investing it in vessel equipment that they can show to customers. This probably makes an IDP the preferred solution to the problem. The independent backup system needs to be easy to locate, initiate, and able to maintain position itself. Non-vessel specific simulator training is of limited use, but a properly designed IDP is ready to go, easy to operate, and a sales feature. Let’s have a look at some of the requirements to make this effective. We will start with the requirements common to both IDP & IJS.
Location, Location, Location: As any real estate agent will tell you, location is primary and the prime location for a DP1 or DP2 IDP, or IJS, is permanently mounted right next to the DP desks and with a good view of position references and the industrial task. Bridge wing and forward bridge control stations are handy for other uses but the IDP’s, and an IJS’s, primary purpose is as an emergency position control system. This means no running 50m to the forward bridge through two normally closed hatches, or other crazy setup. This also means that the owner must not be cheap and use a single portable unit for multiple locations, as people will get lazy and take small chances. Supplementary locations are fine but the DP desk location needs to be permanent and have the operating priority over other IDP/IJS stations at activation. This costs a little more but pays off in reduced risk (always include risks in your budgets). A DP3 IDP might be located in and run through a different fire zone.
Ready: Similarly, the IDP or IJS must be on, running, and healthy, if it is to be effective. It must be easily confirmed to be so and checked before and during redundant DP operation. Considering that every workboat should test its IJS before entering the 500m zone, it is amazing how few people can operate them, and is a clear indicator of industry pressures and problems. This is most easily performed if the IDP/IJS has a large clearly organized screen and controls that are mounted in a convenient location for DPO operation and monitoring, and alarm contacts (not a data link) to the alarm system.
Data Isolation: Data networks are not risk free or redundant, so both IDP & IJS must be independent of data faults that can affect the DP system, and the overall system designed to operate safely if the vessel data systems fail. DP analysts fought with Kongsberg for years to get their IJSs off the network. Ethernet was designed to be fast and good enough, not redundant, and deterministic error handling patches can only reduce some of the underlying system threats. All data systems have significant limits. Kongsberg now recognizes that associated faults can cause an IJS to not be available for use and have generally abandoned the networked IJS for direct connections to the thrusters, but some of their competitors have not yet learned this lesson. IDP/IJS thruster commands, feedbacks, and power limits cannot be dependent on network health and need to be dedicated. DP/Manual/IJS(IDP) mode control cannot be based on a common communication system without creating potential common faults. All DP control system manufacturers have made that mistake at some point and some still do. Shared system and position sensors need isolation modules to prevent a common serial line fault or overvoltage (e.g. lightning) from affecting both the DP and IDP/IJS systems. Some DP incidents give us reason to suspect that some DP systems have common serial line vulnerabilities, but this is not proven. The DP system cannot be allowed to update the IDP/IJS system and corrupt it with its own faults. That is partially why the DP controller changeover sometimes cannot be trusted.
Power Isolation: IDP/IJS operator station, controller, sensor, isolation module, splitter, and mode selection power needs to be independent of DP power, or supplies that can be failed as a result of DP power failure. IDP/IJS can be accepted to fail some shared sensors so long as sufficient sensors are still available to make safe, but a DP power failure cannot be allowed to take IJS control or sensor power. This can be difficult to achieve when both DP UPSs are used to supply all thruster controllers. Theoretically, power protection modules can reliably isolate faults but manufacturing, shipping, installation, maintenance, and environmental faults can degrade these protections without alarm or indication, so separate power is strongly advised. Ideally, an IDP and its subsystems should be fed from a dedicated UPS. Feeding an IDP or IJS from the emergency bus that feeds the DP UPSs means a DP UPS power fault could knock out the emergency supply. Remember that electrical protections are not properly maintained or tested in the offshore, and are far less reliable as a result. A brownout on the IDP/IJS power supply, which was caused by the DP power fault, does not improve the odds of the IDP/IJS working.
Sensors: All IJSs have access to a heading reference, usually from a DP gyrocompass, for heading control, and many have a wind reference, usually from a DP wind sensor, for wind compensation. Some IJSs have access to a position reference sensor and motion reference unit, usually shared DP sensors and usually DGPS, and are capable of non-redundant DP position keeping. As most IJS systems are stripped down DP systems, restoring DP function to an IJS is not usually difficult. An IDP will be slightly different, as it has its own dedicated position reference sensor (probably DGPS), gyro, MRU, and wind sensor, and a slightly different functional priority. These sensors can share information with the DP system, if properly protected against faults, but they are part of the IDP system. A force reference system might replace the MRU and wind sensor and allow improved DP performance but the complexity of the system provides no cost saving.
Selection: Some old IJS systems were very slow to get started. It could take up to a minute to select all the thrusters, sensors, and control modes before getting started and it was hard to read the status on the small, unclear screens. Modern IJSs automatically enable any healthy thrusters when IJS control is selected and screens are much better. Selection of IDP should similarly automatically select healthy thrusters, system references, and position references. I am tempted to have it automatically begin in DP mode and for it to immediately start ramping but, while DP mode is the default emergency operation that we are looking at, there are other modes valuable for non-emergency operation, and the previous thrust state may have been the cause of the position keeping problems. Activation of the IDP should default to DP operation (auto surge/sway/yaw) at the primary control location (e.g. DP desk for DP1/2) but it should be possible to quickly transfer to other control modes in other locations. There might be an argument for a thruster rejection test (deselect a thruster that does not follow commands within tolerance within a significant time) but that could lead to the rejection of degraded thrusters when they are most needed, so emergency stop or deselection must be left to the better informed DPO. Without tighter thruster integration, faults cannot be safely automatically detected. Automatically starting in DP is a major difference between an IDP and an IJS with DP capability.
Model-less: Experienced experts reading this will have noticed a problem. Most DP operation is based on using a DP model but our emergency IDP will start DP operation in an emergency situation with no model. We dare not transfer the model from the normal DP system as a faulty model may have been the cause of the normal DP system’s control failure. The DP model is great for dealing with continuous, uniform, slowly changing situations but much poorer at dealing with discontinuities, step changes, and uneven situations. Obviously, a net force measurement system, like the force reference system that I suggested in my Nov 18/21 DGPS Problems/New Sensor article, could solve the problem by making the model almost moot (an model estimating force is nice but measuring net force is better), but no one currently has one of those for sale or has built an improved DP system by using one. As it is preferred to measure rather than model force, it is preferred to base initial position and heading control on position and heading measurement and previously tested vessel characteristics, until a model is built. This will be a bit analogous to the old fast learn function.
IDP vs. DP3: An IDP is a lot like a DP3 backup DP control system but does not share the DP backup system’s vulnerability to common network and software faults. The cost of this is no model built when IDP first starts controlling vessel position keeping but the benefit of that is no corrupted or incorrect model either. An IDP is a cold standby that is not updated by possibly erroneous information but does not provide a bumpless transfer either, as the thrusters need to ramp back up. This was also true with manual IJS control. DP manufacturers like to claim there is negligible risk from common software, the common network, and the common model, but experience finds that risk to be greater than normally claimed. Many other industries do not consider duplicated hardware, software, and networks to be redundant, but the IDP provides some separation.
Differentiation: Further separation can be gained by using a different vendor for the IDP. It may prove difficult to integrate an outside vendor’s IDP with some proprietary system designs but the advantage is a different set of eyes with different assumptions that are hopefully not making the same mistakes and capable of seeing existing ones in the other system. Type approval does not clear out single point failures but the fear of another vendor finding embarrassing problems might help keep designers on their toes. Another advantage of this is that the industry is less dependent on a single vendor. Kongsberg is a good company but imperfect. What happens when the lawyers, accountants, and MBAs take over and ruin the company? A different IDP manufacturer isn’t necessary but it might help encourage DP industry diversity and quality. On the other hand, a good Kongsberg IDP may act as marketing and increase their overall market share. There is something to be said for more healthy competition.
Counterpoint? There are limits to the effective independence of an IDP or IJS system. IJS independence has been under threat, ever since we started networking thruster controllers. The same hardware and software in the same network environment is not a recipe for redundancy. While most vessels do not have a single type of thruster, it is typical for thruster groups to be provided by the same manufacturer (e.g. bow tunnels, main azimuths) and for a single manufacturer to provide all vessel thrusters. There is ample opportunity for a common fault to lockup all the thrusters. Of course, most designers know this and work hard to reduce the risk of these threats to what they think are acceptable levels. They would argue that they have reduced the calculated risk below the levels of significance for DP (less than the chance of fire and flood for DP2, as safe as two open bus tie breakers for DP3). I can see how that could be true for DP2 but doubt it for DP3. They may be mistaken or may be correct, we can only tell when these proprietary black box system fail. They used to think that this couldn’t happen to DP or a networked IJS. Considering how a potential lockup removes all thrusters from DP/manual/IDP(IJS) control and leaves the vessel without an emergency means of maneuvering, it is safety critical that remote controller resets be available at the DP desk or IDP. This minimum risk mitigation needs dedicated buttons for each thruster controller and each function needs protected against maloperation and electrical faults. For example, covered, monitored, resistance change pushbuttons rather than network commands or unprotected pushbuttons. Every vessel may need these. Smaller vessels, with crew in close proximity to the controllers and familiar with their operation, might be able to do this manually and locally, but those with less convenient layouts, less familiar crew, or crew that are likely to be busy with other work probably should have remote resets. The best practice would be to default to remote manual resets. Automatic remote resets may introduce failures that fail multiple thrusters and would just change the risk source.
Other Threats: Poorly designed, implemented, or maintained power management or power generation systems are also a threat to IDP or IJS independence, but I will leave that large topic for future articles. It is possible to keep going and start talking about fuel handling, lube oil, cooling, etc. but that is going too deep into overall redundancy and away from our limited topic. These things affect the independence of the systems that the IDP or IJS control, and can be considered separately.
Conclusion: IJS isn’t really working for us due to various industry pressures squeezing operators. A hardware fix is more likely to be adopted and effective than a return to previous human based solutions. Some operators already have IJS systems with DP capability and most IJS designs are DP capable, so further refinement of the IJS design to produce an IDP will require little work. Use of this improved product should improve DP control system failure response, provide increased client assurance, and be an advantage in vessel marketing. If DPOs no longer have the time to know how to work IJSs then IJSs need replaced with IDP systems that can do the work.