Security and Resilience (Cyber and Physical)

Cyber resilience: Preventing a Ukraine-style utility attack in the U.S.

Keeping the electric grid secure takes commitment from many sources — utility staff and leaders, third-party vendors, and state and federal government. While technology plays a part, experts cite a variety of nontechnical factors that strengthen a utility’s cyber defenses.

Addressing the human factor

According to Kevin Johnston, ITS security architect at Snohomish County Public Utility District in Washington, cybersecurity starts at the human level. For example, Johnston explained how a recent attack on an electric utility by the Dragonfly group used social engineering tactics. One of Dragonfly’s tactics was a “watering hole” attack, where the group identified common vendors and industry groups used by many hundreds of utilities, and then infected those entities’ networks to access user credentials.

“The tactic there is to hide under the trusted relationship that has already been formed,” explained Johnston. “They’re not attacking the firewall protections on the network, they’re not trying to brute-force their way in through an active technical control. They’re simply exploiting the emotions and behavior of the human users.”

That’s why knowledge is key.

“Training is fundamental to any cybersecurity effort; without having employees trained in cybersecurity skills, hygiene, and data protection, and basic rules and proper behavior on the network, any cybersecurity program is immediately threatened,” said Johnston.

“When real-world operators get shown the ease at which their technical controls can be penetrated by a complex and well-executed social engineering campaign, it really brings the point home,” he said.

That’s why Snohomish invited the National Guard to attempt to hack the utility in a hands-on exercise. Being in Washington state means that members of the guard are people from technology companies such as T-Mobile, Microsoft, and Facebook. “We have the cream of the crop,” said Jessica Matlock, director of government, external affairs and strategic accounts at Snohomish PUD.

“We were paying attention to our firewall. We were paying attention at a higher level,” said Matlock. In the exercise, members of the guard hacked the PUD’s system. “[They] told us, your firewall was protected, but we decided to go for the easy route. And the easy route was through a phishing email.”

Matlock said the exercise opened their eyes to the importance of basic cyber hygiene, such as training people to watch out for phishing emails or tightening administrative access.

Taking security seriously

“Utilities are 100 percent on their own for security,” said Robert Lee, CEO and founder of security firm Dragos. He acknowledged that there is a misconception about when federal agencies will provide support to a utility that has experienced a cyber incident. “What the government is defining as an attack is different than what utilities define as an attack. What the national leaders actually intend is when a conflict-scale attack occurs, that’s when they’re willing to come in. They’re not really intending for incident response teams to come in from the government and do assessments and technical work,” he added.

Tim Conway, an instructor with the SANS Institute, noted that throughout his career, smaller entities have often voiced the opinion that they are too small to ever be the target of an attack from a big adversary group. “They don’t get to tell the adversary groups or nation-states, ‘This organization is too small; don’t mess with them.’ It’s an adversary’s decision to make,” said Conway.

Conway sees a parallel to the attacks that happened to Ukraine’s electric system.

“In 2015, the adversary could have chosen to target transmission facilities, but targeting a large national asset might have demanded an escalated response. They could have gone after other countries, but that possibly would have dragged in NATO and some other additional response. Targeting distribution-only organizations might fit the adversary’s objective in sending a signal to the world while limiting the electric system impact,” he explained. “The same thing could happen here in the United States. Rather than going after some of our largest IOUs, which would demand an aggressive level of response, going after a smaller PUD or smaller utility would send the message across but may not demand a massive, full-scale response from the government.”

Matlock stressed that other utilities could take advantage of a partnership with the local National Guard for assistance for more involved attacks. “They can then be deployed through the governor in the case of an attack through the state, and they can go in and assist utilities like ours,” she said. Because the PUD already built the relationship and went through the exercise, Matlock noted how this help could be more efficient since the guard is already familiar with the operating system.

Practical steps to protection

“Historically, the utility industry has been really good about the process piece and management piece. What has been missing is the technical component. Even having industrial-specific technologies is a new thing in the industrial control community,” said Lee.

“Utilities have invested heavily in security, but not necessarily in operations technology-specific security,” he said. “Every utility has a response plan, but actually not many of them have an industrial-specific one that takes into consideration the industrial environment and the different threats and scenarios that might come into play.”

Network segmentation across the information technology and operational technology systems is a fundamental security practice. “But just because you have a well-segmented system does not mean you are completely protected. Once the credentials are compromised, within the segmentation, then lateral movement is possible,” said Johnston. He pointed out that the Department of Homeland Security’s recent report on utility attacks showed that attackers compromised credentials from corporate networks to gain access to the SCADA system.

“There is no true ability to segment these environments because of the business requirements to pull data in and out of them, as well as the operations requirements to do the same,” said Lee.

Nevertheless, Lee says utilities should have some systems segmentation in place, as well as the ability to understand, detect, and respond to threats.

“If you can get ahead of the adversary, great, but you shouldn’t be falling prey to attacks that are well-documented,” said Lee.

According to Ben Miller, director of threat operations at Dragos, the most common path an adversary will try to take to get to a utility’s SCADA system would be through an IT network. And that IT network does not have to be internal to the utility, but could be a third-party vendor’s system or through a remote access capability, or a technician who brings his or her own laptop or other devices to interface with the utility’s equipment.

Miller suggests that utilities reduce risk from these sources by having security teams track or scan any USB drives or other removable media. 

Miller said utilities should also be aware of which devices are connected to each part of the system and be able to isolate when something changes in the system. This visibility makes it easier for a utility to know when something goes wrong and doesn’t allow threats to passively linger inside your environment.

Lee pointed out that a lot of what utilities want to protect is no longer within the utility environment. Utilities should lay the framework for vendor security expectations at the outset through specific contract language.

“What we normally see is language such as if the vendor detects that your data has been compromised, that they must inform you, but that’s actually requiring the vendor to know way too much about their own incident before they can warn you about the risk they’ve given to you,” said Lee. “It shouldn’t be that they must prove that your data has been compromised to inform you. It should be the opposite, which is, if there’s any indication that it could have been, they must inform you.”

From regulation to implementation

The litany of cybersecurity guidelines and federal and state regulations should be a foundation that utilities build on.

“The [National Institute of Standards and Technology] guidelines are the best cyber roadmap in terms of protecting and hardening your network. The recipe has been provided by the federal government, [from] lessons that have been hard learned,” said Johnston.

However, “The NIST standards are standards. They are just the basics. Regulation is not protection,” said Matlock. “Because these cyber attacks are changing day in and day out. How can you have a regulation that protects you when threats are evolving on a daily basis?”

“You have a standard regulatory lag that is drifting behind threats and drifting behind cyber technology,” concurred Conway. “And then you have the implementation lag. Asset owners and operators aren’t the ones who make this equipment or write this software in most cases. They have a small vendor pool that they can pull from and they have to wait for the vendors to integrate some of the capabilities spelled out in the standards.”

“A lot of the regulations and compliance written to date have been copy and paste from best practices out of enterprise IT security,” said Lee. “We’ll not only need to have the tech and approaches specific to our actual threats, but we’ll need to be flexible about how we treat the compliance and regulation to make sure we can update it based off of what we’re learning instead of being rigid.”

Conway trains organizations on how to build from foundations such as the North American Electric Reliability Corporation’s Critical Infrastructure Protection standards. “We know anyone can pick up the standards and read them; we’re not just focused on teaching you the standards, we’re teaching you how to implement a technical program and maintain it,” said Conway.

Cybersecurity is not last on the list

“When we race to implement something, it is all focused on getting that into the revenue stream, to build an asset and get it going as fast as possible,” said Conway. “Cybersecurity often comes last.”

He mentioned utilities should look at large technology deployments, like smart meters or syncrophasors, to ensure they are not taking an approach of “dealing with cybersecurity issues later.” He is working with utilities to incorporate cybersecurity into new projects or upgrades earlier in the process.

Just as utility projects conduct a factory acceptance test to ensure equipment meets operational standards, they could be testing to ensure cybersecurity. “We’ve been teaching that you really need a cyber acceptance test at various stages of a project validation, to ensure that [new technologies] meet your cyber requirements,” Conway said.

In addition to the training and exercises, the Snohomish PUD conducted a cyber risk assessment to identify the principal sources and likely damages of attacks as a way to forecast potential financial losses. Matlock described this process as similar to the way health insurance companies determine premiums, which in this case involves modeling threats in the form of attack campaigns against a network and forecasting the most likely successful attack pathways to determine risk of loss.

“Not only did it help us patch up areas, but it helped us figure out what our priorities are, how to build our cyber budget, and then convince our CEO and commission what was important and what we needed for funding,” said Matlock.  

Know what you have and how to use it

“A lot of time, what security comes down to is good management and good visibility,” said Miller.

“Whether you have an enterprise of 30 applications or 300-plus, properly understanding the what, why, and how of what is on your network is essential,” said Johnston.

That means business and IT leadership should sit down regularly to review what is on the network, why it is on the network, and if that system is still necessary. Johnston said this approach can help prevent utilities from getting new applications, hardware, or software without sunsetting or removing the legacy applications and software.

“In those situations, networks become overburdened with useless applications, which are just opportunities for entry,” he added.

Conway also noted how utilities often might find funding to buy gear and equipment for cybersecurity systems, but then don’t have the staff to maintain those systems. “Without necessary care and feeding, these new cyber assets don’t work well. Those systems then die out, and new cyber assets are pursued, which repeats the cycle. Utilities need to focus on having knowledgeable, trained staff first, then obtaining the tools and technology,” he said.