NIST Framework Misses the Mark on Risk Assessment

Last week, the open enrollment period ended on the latest National Institute of Standards and Technology (NIST) preliminary cybersecurity framework. While changes are likely, given the input of many in the industrial sector, the framework as it stands does not meet one of its key goals – it is not simple enough for senior business leaders, executives and board members to understand and implement to achieve measurable impact on security efforts. In particular, Waterfall Security’s feedback into the framework process points out that the framework has missed an opportunity to encourage utilities to rethink their approach toward risk assessment, and points out that the framework leads senior management to ask the wrong kinds of questions about the security of critical infrastructure sites.      

Capabilities versus actuarial

Risk assessment is notoriously subjective in that businesses routinely conclude whatever they want and point to a “risk assessment process” to justify their decisions. Even worse, a major issue with the NIST framework is that it encourages an actuarial approach to risk assessment for determining what, if any, improvements need to be made to a security program. This methodology compares the number and cost of known incidents to the total cost of a security program which would have been strong enough to prevent these incidents. If historical costs are small, insurance can be purchased to transfer the risk.

Contrast this with a capabilities-based risk assessment, which compares our enemies’ attack capabilities with our own defensive capabilities, and asks when the one meets the other, what is the most likely outcome? This kind of assessment does not ask how many times was the North American power grid taken down by a cyber assault in the last decade, and what did each such incident cost? The answer is of course, zero – the power grid as a whole has never been successfully attacked. A capabilities-based assessment asks only when our most capable enemies attack us, what is the most likely outcome?

The big difference in assessment techniques is due to the increasingly widespread understanding of “new, advanced attack techniques.” These techniques by now are neither new nor particularly advanced – they are taught in all intermediate-level cyber security training programs. Everyone with a little skill knows how to apply these techniques. Existing “risk-based” security programs at industrial sites, including sites throughout most of the power grid, are inadequate to attack this “new” class of attack. For example, security testers who use these techniques routinely report breaching security at fully CIP-compliant sites within minutes of launching their tests. Deployed defensive capabilities have not kept up with well-known attack capabilities.

No – there has not yet been a significant, well-documented attack on the safety or reliability of critical infrastructure sites using these new techniques. But using the actuarial approach and saying “it’s never happened so I’ll just buy insurance” is seriously misguided. Can you imagine the CEO of a large power utility on television after some cyber attack causes a major power outage that casts tens of millions of people into darkness for several days? Can you imagine this person saying “We are very sorry this attack that made all your power go out for so long – but never fear, my utility did not lose any money, because we have insurance.” He would be lynched.

Advanced Defenses

Now, the truth is often more complex than an “actuarial vs. capabilities-based” buzz-word. For example, many business-focused IT experts are on record saying that they are no longer confident that even the best security programs can block these “new advanced attacks” at the network perimeter, and so these experts are advocating the deployment of sophisticated intrusion detection, data exfiltration prevention and other systems.

This assessment may be accurate for corporate networks, but the NIST framework is supposed to speak to critical infrastructure control system networks and the “SCADA Security” problem primarily, not corporate information protection systems. A variety of control system-centric security technologies have proven very effective at blocking even “new advanced attacks,” not least of which is Unidirectional Security Gateways. This hardware-enforced server replication technology completely blocks the interactive remote control capabilities which are essential to “new advanced” attacks – a fact which is not called out in the least by the NIST framework.

Capabilities versus motives

In all fairness though, NIST’s risk management approach does use the word “capability” a couple of times, but only in wholly secondary contexts. The focus of the risk management part of the framework is still an actuarial “cost of incidents” style of risk assessment. No, there have not been credible cyber-sabotage attacks on critical infrastructure sites using modern, well-known attack techniques. But  plugging this fact into an actuarial risk calculation is not going to produce a security program which protects against our adversaries’ capabilities, it will only produce a program that protects against what we think our enemies’ motives are. If we guess wrong as to those motives, we are in deep trouble.

We need to start communicating more effectively to senior decision makers. We need to persuade them to invest in security programs which anticipate our enemies’ capabilities, not programs which hope we can guess right as to our enemies’ motives.

To stay updated on new developments follow Waterfall Security on Twitter.