How do you measure Cyber Resilience?

Preamble

These days ‘cyber resilience’ is a term that is used by almost everyone – starting from government officials talking about cyber security strategy, cyber security guidelines, cyber security architecture, to CISOs of organizations exhorting his/her team to build an IT or IT/OT system that is resilient to cyber-attacks, all the way to academics who often interchangeably use the term to describe the ultimate goal to achieve in cyber security posture.

While the term is over-used, thrown around like a buzz word, and often used to indicate something very lofty to attain, I find that the term is overloaded and often misunderstood. Further, no one seems to have an answer for how to know when one has achieved cyber-resiliency? Are there degrees of cyber resiliency? Is there a numeric or categorical measure of this property of a digitalized organization?

In this short write up, I try to settle this question – without being too pedantic but rather from a pragmatic perspective.

Definition of a cyber security state of an organization

In my view, every system has a cyber security state. If a system is designed with a threat model that is excessively permissive with no firewall, no anti-virus, no intrusion detection and very primitive authentication and access control (e.g., plain old password protection without any rules for password creation, longevity, or encryption) – the system is in a state of zero cyber security. An cyber attack on such system is extremely easy.

Once the system is equipped with certain cyber security control without having a well-documented, well informed threat model, without risk assessment and risk-based control, then the system is in an ad hoc cyber security state – where an ad hoc list of guidelines has been followed but not a risk-aware, risk appropriate control mechanism, no monitoring or monitoring based response mechanism in place. Such a state can be seen in Indian academic institutes including the IITs.

Organizations that follow risk-driven standards as guidance or compliance requirement for their cyber security, they would assess risk at least at a high level, put in controls to mitigate risks identified, would monitor their network and endpoints, do periodic vulnerability assessment, and have incident response mechanisms, would have disaster recovery provisions etc. Such a cyber security state, I would call risk-informed.

A yet better state from risk-informed state would be when risk measures are periodic or triggered by changes or events, when continuous monitoring, response, and risk re-evaluation is common, when there is recovery mechanism not only in terms of provisions but through periodic recovery drills as an organizational routine, and when the effects of cyber security controls are measured very seriously and often – we say that such a state is risk-driven cyber security state. Such a state also means periodic audits for not only required standards by regulators but also other standards such as NIST standards for supply chain security are also complied with.

Another state beyond or above this is the adaptive cyber security state which not only has achieved all that is achieved in a risk-driven state, but also adapts automatically or semi-automatically with changing threat landscape, changing attack patterns, newer types of attacks, adversaries etc. Such systems adapt to organizational changes such as large-scale network overhaul, addition or subtraction of large part of the network due to acquisition/merger or splits. Such a state when achieved can provide a CISO and the board of the company a lot of confidence about its ability to not only protect, detect, respond and recover from large- or small-scale attacks but also from novel attacks of various kinds. Some of the events they need to recover from may not even be adversarial such as acquisition/merger/split events that may bring large changes in the system architecture, network, and man power.

One could think of an organization’s cyber security states to be more nuanced, and even within each of the states I have described, there could be embedded state machines – at various levels of completion towards achieving what I have described. However, that is not the point here.

In parallel to the cyber security state machine, we can also imagine a state machine of functional capacity and performance. If a system fails to provide minimum functionality and minimum level of performance, we can include all such low-capacity states into a failed state. Beyond this, one should be able to define gradually improving sequence of states up to a state where the maximum possible functionality and performance is obtainable.

These state machines run in parallel and have dependence. If you are in a zero level cyber security state, and in the maximum functional state, a cyber attack can sharply bring your system down to failed state. However, if you are in the best of cyber security states, then a cyber attack may temporarily bring you down to a lower functionality state but you are likely to move back to full functional state autonomously or with human intervention. Whether that will take time, whether this return to a desired state will be intervened by a sequence of lower functionality states or directly move back to the desired state will depend on your resiliency.

A Definition of Cyber Resiliency of an Organization

Resiliency is neither reliability nor robustness. Reliability is defined as the probability of guaranteeing a certain property. If maintaining one of the above cyber security states is a property one has targeted, then reliability would be a measured by how often in the face of adversarial actions, would the system NOT lapse into a lower state, or collapse into a state where minimal functional and/or performance cannot be provided.

Robustness is defined as the ability to perform functionalities at the minimum required performance level in the face of adversity. If it lapses into a lower state of functionality – and a lower state of cyber security – we shall consider poor robustness.

Resilience on the other hand is the ability to come back to the required state of functionality/performance from a compromised state of functionality. In case the compromise happened due to a cyber-attack, and it is the cyber security state you are in determine how fast or slow you will recover to the desired state.

So how do we measure Cyber Resiliency?

One could measure it in many different ways – for example – from degraded functional/performance state S_d, how many intermediate functional/performance states are to be visited before recovery to full functional state S_t. However, most organizations will not have these states documented and these states are implicit. Therefore, using the number of such states will be difficult.

What then can be measured? A good measure will be time it takes to go from degraded state S_d to target functional state S_t.

How do we measure the time? Of course, if the system has had suffered many attacks in the past which led to degraded state and recovery had to be done – one could statistically measure the time. However, most systems do not go through such attacks especially if the system is also in its highest (adaptive) state of cyber security. In such case, one has to estimate this time.

Now, the time required will depend on our old friends – people, process and technology.

For example, if my system suffers a ransomware attack even when my system’s cyber security state is adaptive and passes all audits, and risk assessment showed we are at a below tolerable risk level – are my people, process, technology orchestration has been arranged so well, that I recover in no time?

This means do I have latest backup of everything including applications, data, operating system, firmware, access control list etc? Have I had my people do frequent drill on how to launch system recovery from such backups? Are the backups protected by data integrity code? Was the backup storage disconnected to save them from the tentacles of the worming ransomware? Do I have the technology to wipe out ransomware that may hide in boot sector? If I have all those according to a recovery check list, I may be able to recover in minutes to hours.

Using these guidelines, we can create checklists and benchmark against the partial or complete satisfaction of the items on the checklist to estimate the time to recover to full functional state.

Conclusion

It is my considered view that in order to measure resilience, we have to first plan and document the recovery process, people responsible, backup people in case of absence of key people, the technology readiness to recover from backup with necessary tools and techniques. This plan and documentation will be dependent on the system, its functionality and performance requirements, its architecture, network, and its cyber security state. This is a nontrivial research problem to come up with such a methodology and checklist – but not impossible.