Remote Access Best Practices

Common wisdom is that “if I have a firewall and encryption, I must be safe.” Virtual private networks (VPNs) are seen as the solution to the remote access problem, but this common belief is very much mistaken. Viruses, malware and online attacks move through encrypted VPN connections as easily as they can move through un-encrypted local area networks (LANs). The whole point of a VPN is to make remote users feel as though they are locally connected to trusted LANs. Encryption provides protection against data theft, data manipulation, and man-in-the-middle attacks, but it provides zero protection against attacks from either the networks they connect, their workstations, or their endpoints. Laptops, workstations and mobile devices used for remote access are notoriously prone to compromise.

Now to be fair, understanding of remote access risks varies greatly. Some utilities very much do “get it” and have deployed powerful remote access protections. The Department of Homeland Security (DHS) Catalog of Control System Security: Recommendations for Standards Developers provides much better advice than just “use a firewall and VPN.” The North American Electric Reliability Corporation (NERC) 2011 Guidance for Secure Interactive Remote Access is even better – in fact, it’s pretty good, as it even starts to mention more advanced and appropriate protections.

One of these more advanced protections are hardware-enforced industrial cyberperimeters, which are Waterfall’s focus. All software has bugs, and some bugs are security vulnerabilities. In practice then, all software is vulnerable. The Heartbleed bug and second set of OpenSSL bugs makes this point in spades. These bugs allowed attackers to steal private security keys from public-key cryptosystems used by a large fraction of the world’s websites, and in fact used by OpenSSL-based VPN implementations, as well. Since the bugs were announced, I have spoken to many experts about them. Not one believes that this vulnerability lay in wait, un-exploited, these last many years. Governments and organized crime rings all over the world have spent billions over the last decade to develop sophisticated attack tools, and to find and exploit zero-day attacks. For example, the latest Wikileaks revelation is that the NSA has a list of vulnerabilities able to compromise pretty much every firewall in existence.

All software is vulnerable. Software protections have failed repeatedly to protect IT networks. Why, then, should we trust software to protect critical control system networks, especially when hardware-based protections are available? Unidirectional Gateways replicate industrial servers for painless, safe and continuous remote monitoring. Remote Screen View lets remote personnel see the screens of critical machines, and participate in emergency problem resolution by directing the actions of local personnel over the phone in real time. Even a simple Secure Bypass device adds enormous value to an emergency VPN capability. No remote user should have the power to initiate a remote connection into a protected, critical network without the knowledge and participation of personnel at the industrial site. And no targeted attacker should have the power to initiate a remote connection to an industrial site simply by attacking software.

Hardware-based remote access protections are more powerful than software-based protections, and are far simpler. Serious software-based protection is not easy at all, and as hard as it is to implement, those protections can never be as thorough as simple hardware-based protections.

The time for hardware-based protections has arrived. Why is everyone still talking about software?

For more information on products for protecting your critical infrastructure site from the Heartbleed vulnerability and other remote access pain points, please click here.

The Security Risks of Remote Access and “Cloud Control Systems”

Today, Waterfall Security has the privilege of participating in a panel at the Federal Energy Regulatory Commission (FERC) Technical Conference to discuss technical and operational issues in the North American Electric Reliability Corporation (NERC) Critical Infrastructure Protection (CIP) Version 5 standards that the commission approved in late 2013. While Waterfall supports the improvements made in NERC CIP Version 5 over previous versions, we have expressed concerns over the cybersecurity of central turbine monitoring systems and other types of interactive remote access. 

Most turbine vendors require continuous monitoring of operations as a condition of hardware warrantees and hardware support contracts. Turbine vendors generally also require that sites allow occasional remote control of the turbines to correct vibration and other anomalies that might grow into catastrophic failures. This continuous monitoring and occasional remote access is most often enabled by an encrypted virtual private network (VPN) tunnel connected directly from the central monitoring site to the turbine owner’s control system network. This tunnel generally bypasses all of the owner’s corporate security technologies. This means is that there is only one layer of security between any vendor-monitored turbine anywhere in the world – even those turbines not under the jurisdiction of NERC CIP – and any vendor-monitored turbine in the U.S. Further, the remote access permissions that sites grant to vendor monitoring and diagnostics centers are much more intrusive than the Inter-Control Center Communications Protocol (ICCP) set point permissions that are typically granted by generating utilities to BES Control Centers. This makes no sense though. The CIP Version 5 standards require many technical security controls for BES Control Centers, but the central vendor sites are not held to the same standards. In fact, there are currently no technical security measures required of central vendor sites.

These concerns reflect a deeper issue around interactive remote access from any one site or laptop to a large number of similar targets in the Bulk Electric System (BES). In many utilities, there is a set of senior engineers’ laptops which are configured to with “occasional” on-demand access to any of hundreds of generating and substation/switching sites via VPN. In addition, many control system technology vendors are setting central “monitoring and diagnostics” sites for their products, not just turbine vendors. While individual remote access is a security problem, should an attacker take advantage of an opening, these risks are multiplied when one site or one laptop has access to hundreds of such VPN connections.

This issue is easily resolved through a combination of hardware-enforced Unidirectional Security Gateways configured with unidirectional Remote Screen View capabilities. Since the gateways enable one-way communication channels out of the control system, there is zero risk of a cyberattack getting back in. Vendors who need to fix anomalies can call the affected site and turn on screen sharing through Remote Screen View to see site screens and to direct engineering personnel to correct the problems.

Waterfall has formally submitted these comments in a prepared statement for the April 29 FERC Technical Conference. It is our hope that FERC and NERC will take a deeper look into the risks posed by central turbine monitoring and remote access systems, and will take the appropriate measures to mitigate these threats to the reliability of the BES.

What the end of Windows XP support means for industrial cybersecurity

We are now in the final month of Microsoft offering support for its Windows XP operating system, which presents a new security challenge for the great many control systems still running XP. Without support, XP control systems will not receive regular security updates, making them susceptible to cyberattacks. Control systems running older versions of XP will of course be no less secure than they already are.

This shouldn’t be news to any utilities using Windows XP, as Microsoft is pretty transparent about the Windows lifecycle. (Set your calendars now for April 11, 2017, Windows Vista users, however few of you are.) Despite knowing that support is ending, laggards among control system vendors are still shipping new products on Windows XP, demonstrating an “if it ain’t broke, don’t fix it” attitude. Well, Windows XP is now officially broken.

Utilities aren’t exactly early adopters when it comes to new operating systems — and with good reason. With every new operating system comes a host of bugs and glitches that put reliability-critical and safety-critical systems at risk. When Windows 8 was released, the control system world watched and learned as corporate information technology (IT) teams struggled with it while the kinks were ironed out. Only once a technology is proven and the reliability risks well-understood do we start seeing industrial customers begin to deploy the technology. This shaves at least a few years off the lifespan of operating systems in the industrial world compared with the corporate world.

This has long been a problem with no simple solution and reflects a larger debate surrounding the issue. Upgrading an industrial control system to the latest operating system is generally impossible, as the old version of software generally does not run the same (or run at all) on a new operating system. Regularly upgrading to new versions of control system software is often cost-prohibitive, due to the resources needed to test a change that big. The testing cost of installing regular security updates at all is prohibitive in complex environments with serious safety and reliability concerns.

For the foreseeable future, and very possibly indefinitely, a great many control systems will continue to suffer from a very “soft interior” security-wise. Compensating measures in the form of strong physical security perimeters and strong cybersecurity perimeters continue to be far more important in preventing attacks to control system networks than these measures are important to corporate IT networks. One compensating measure we see being deployed ever more widely is hardware-enforced Unidirectional Security Gateways, which allow business-critical industrial data to flow in one direction out of a protected network, without any chance of an attack getting back in through the equipment.

The day is upon us. If our control system has a soft interior, we had better put a hard shell around that interior if we want to stay safe.

Read more about how Unidirectional Security Gateways can protect critical infrastructures.    

Desperately Seeking SCADA

Shodan, “the scariest search engine on the Internet,” was back in the news this month with the launch of Shodan Maps. For those unfamiliar, Shodan tracks devices that are connected to the Internet, including SCADA and industrial control systems (ICS). Now, instead of just identifying these systems, searchers can see where they’re located. This is troubling, as it gives our adversaries physical directions to what appear to be poorly defended critical infrastructure systems. 

Fortunately, Shodan isn’t designed for your average Googler. Those who are capable of carrying out a large-scale cyberattack against critical infrastructure sites, though, will have the technological knowhow to navigate the search engine. Researchers with Project SHINE have identified more than 1 million IP addresses globally that are potentially associated with SCADA and ICS devices. However, at the recent Public Safety Canada ICS Security Workshop, it was reported that the DHS investigated the 500,000 American IP addresses SHINE reported, and found that only a little more than 7,000 were real control system equipment. While this is a small percentage of the original number, it is still a disturbing amount of equipment.

The issue remains: in a constantly connected universe, any system that is connected directly or indirectly to the Internet is vulnerable to attack. Large scale control systems recognize this and are buried behind layers of firewalls, but firewalls aren’t enough to defend against modern day cyberthreats. Firewall vulnerabilities are well known to anyone with a modest security background, and control systems connected to the Internet is a problem made worse by exposing them via search engine.

The best-defended control systems, such as those at every American nuclear plant and an even larger number of conventional power plants, have installed Unidirectional Security Gateways, a stronger-than-firewall technology that thoroughly protects control systems from Internet attacks, however indirect they are. That someone with average skills can locate Internet-exposed control systems should inspire any utility manager to improve defenses.

See how unidirectional security gateways can deliver true security.

Follow us on Twitter @WaterfallSecure.

Like us on Facebook.

Follow us on LinkedIn.

S4 Takeaway: Differences between North America and Europe

I've had a week now to reflect on my experience at Digital Bond's 2014 S4 conference and compare it to others I had last quarter. I spent almost all of Q4 2013 on the road with customers and at conferences, both in North America and Europe. While it is unfair to paint either region with one brush, I do see broad differences in the level of understanding of industrial cybersecurity issues.

For example, in North America, I rarely have to explain why cybersecurity is important to industrial sites. I am often asked to relate what the latest developments are, how they fit into the overall risk and solution picture, and how various debates in the industrial control system (ICS) security field are evolving. I also tend to see people working to understand how the latest developments fit into their understanding, and deciding what these developments suggest as to how organizations need to evolve their security programs.

Compare that with Europe, where interactions with representatives of critical infrastructure owners and operators tend to be more challenging. Operations teams will often say that security is an IT problem. However, IT is busy deploying privacy-focused security technologies on operational and control system networks with limited success. Both groups struggle to see how state-of-the-practice OT security technology, like Unidirectional Gateways, fit in their IT-style integrated systems and IT security programs. The widely known, very effective techniques used to break into IT-style networks are a risk Europeans tend to discount.

In hindsight, my experience at S4 really highlighted these differences. The programs and discussions were all about control systems and issues and solutions specific to them. For example, I'd been hearing about the DNP3 news second-hand, and was really looking forward to Chris Sistrunk's presentation of his and Adam Crain's work. "Ahh," I thought, "they used a fuzzer! And against master stations, not remote terminal units. Of course they found vulnerabilities – good for them! More work like that needs to be done." And then there was the "risk-based" debate regarding the NIST cybersecurity framework, which crystallized the understanding that cyber risk assessment really does need to be different for critical infrastructure sites versus other kinds of industrial sites.

To be fair, there are other large (by ICS cybersecurity standards) and high-quality events on the slate every year in North America – NERC's GridSecCon, the SANS SCADA events, the ACS Cyber Security Conference and the DHS ICSJWG events all spring to mind. Many Europeans attend and contribute to these events and to S4 – the real leaders in Europe are no slackers. But S4 seems to be where the world's technical leaders get together to argue things out. Differences between experts remain when the day is over, of course, but at least everyone goes away armed with the latest data and counter-arguments.

Admittedly, my experience in Europe is limited. I've only just started attending events and visiting European customers, and there is a spectrum of sophistication in a region this large and diverse. For example, one meeting I had with senior government and industry leaders in the UK earlier this year was impressive. I did not have to explain to them why security was important or how control systems were different – they were the ones asking me the hard questions, testing my answers and seeing how my information fit into their understandings and plans.

On average though, I have the impression that there is more awareness-level work to do in Europe than there is in North America. Look for greater Waterfall participation in European industrial cyber-security events, as well as standards, guidance and research efforts.

By the way: business is booming. We're swamped. Waterfall is hiring. Not all the jobs are posted yet. Drop us a note if you enjoy working hard, doing important work and being on the road a lot:  jobs@waterfall-security.com

Cyberwarfare is a bigger threat to U.S. than terrorism – now what?

Defense News released last week the results of a poll in which nearly half of the U.S. national security leaders who responded cited cyberwarfare as the most serious threat facing the country, ranking it higher than terrorism or China. The alarm bells have been ringing for some time now – the sound got louder last year, when President Obama issued an executive order to update the National Institute of Standards and Technology (NIST) cybersecurity framework. With this latest survey it seems that a consensus is starting to emerge that yes, cyberattacks are a real threat and something must be done to better secure the nation’s critical infrastructures.  While it’s encouraging to see this issue finally getting the attention it deserves, the question still remains “will anything will be done about it?”

Unfortunately, it will probably take a large-scale attack for some utilities to get serious about improving security defenses. This is because many utilities base their risk models on the likelihood of an attack – and without a significant event to reference, the probability of future attack must be low, right? In fact, all it takes is one significant event to trigger the risk models, and no utility wants to become the new poster child for critical infrastructure cyberattacks – one that will be talked about and analyzed for years to come. While the first big attack will almost certainly light a fire underneath utilities, this isn’t something we can afford to wait for when power, or clean water, or clean air for millions of people are at risk.

There is progress to report. The North American Electric Reliability Corporation (NERC) recently updated its Critical Infrastructure Protection (CIP) standards to Version 5, which is a marked improvement over Versions 3 and 4. These standards will go a long way in bringing power grid security up to par by encouraging the adoption of new technologies that are stronger than firewalls.

Firewalls are no match for the advanced level of today’s cyber threats. It’s like going into battle using paper clips for armor. Years ago, a lot of us were confident that a firewall would block the vast majority of cyberattacks, whether we were right or not. Nowadays, firewall limitations are well-known in both white-hat and black-hat communities. The problem is that security practitioners in utilities often have trouble communicating this risk to the management teams who control security budgets. These teams often don’t understand just how poor their defenses are until someone shows them how easy it is to breach those defenses. Hiring a penetration tester is a good way to expose poor defenses, before our enemies do.

NERC has recognized the value of Unidirectional Security Gateways. What will it take to communicate the risk to other decision-makers? Sometimes what it takes is a security breach – malicious or benign – to change our thinking. Better a white-hat penetration-testing breach, than waiting to become a poster child for a black-hat cyberattack. 

Strategic Defence Intelligence recognizes Waterfall Security’s achievements in protecting critical infrastructure

There’s a familiar name in the Strategic Defence Intelligence and Global Defence Technology Awards – Ones to Watch 2013: Waterfall Security. The awards recognize outstanding achievements in defense technology and innovation, validating Waterfall’s unidirectional security gateway technology as stronger than firewalls and underscoring the potential for protecting our nation’s critical infrastructure.

In its announcement of the shortlist of recipients, Strategic Defence Intelligence says that:

“Waterfall Security’s Unidirectional Security Gateway solutions offer effective protection for safety-critical and reliability-critical networks. Unlike competitors that specialise in military and government requirements only, Waterfall’s products cater extensively to the industrial space as well, important when you consider that some of the most sophisticated attacks emanate from outside conventional warfare.”

Highlighted in the assessment of Waterfall’s solutions is our Waterfall for Bulk Electric System (BES) Control Centers, protecting two-way communications via hardware-enforced Unidirectional Security Gateways replicating inter-control-center protocol (ICCP) communications endpoints in two directions.

Last year saw renewed focus in the vulnerability of our nation’s critical industrial infrastructures to cyberattacks. In February, President Barack Obama signed an executive order requiring the National Institute of Standards and Technology (NIST) to create a cybersecurity framework, a preliminary version of which was announced this past fall. (See our take on the framework here.) Perhaps the biggest news of 2013 was the Federal Energy Regulatory Commission (FERC) approval of the North American Electric Reliability Corporation (NERC) Critical Infrastructure Protection (CIP) Version 5 standards, replacing the ineffectual Version 4 and allowing utilities to move forward with strengthening security programs. 

While there is still a lot of work to be done, we are optimistic that 2014 will see improvements made in securing the nation’s power grids and other critical industrial infrastructures. The more we talk, the more advanced our adversaries become. Firewalls are no match, and it’s time to take action to better secure our infrastructures.