Cybersecurity and Physical Security

Grid exercise evolves as public power participation increases

A grid security exercise designed for utilities to exercise their response and recovery to cyber and physical security threats in a simulated environment has evolved in several key ways, while the participation of public power utilities increased for this year’s exercise, key government and industry exercise participants said on Nov. 14.

GridEx, which takes place every two years, allows utilities, government partners and other critical infrastructure participants to engage with local and regional first responders, exercise cross-sector impacts, improve unity of messages and communication, identify lessons learned and engage senior leadership.

The exercise began in 2011 and the North American Electric Reliability Corporation hosts the GridEx series. The 2019 GridEx, which started on Nov. 13, marks the fifth such exercise.

In a conference call with reporters on Nov. 14, Jim Robb, President and CEO of NERC, noted that the exercise’s participation rate continues to grow. He said that more than 425 organizations took part in the exercise this year “and importantly increased participation by the public power sector and the rural co-ops.”

“Public power utilities are increasingly realizing that while they are typically smaller than their investor-owned brethren they are not immune to cyber and physical threats,” said Nathan Mitchell, Senior Director of Cyber and Physical Security Services at the Association.

“We at the Association, in partnership with the Department of Energy, have elevated cyber and physical security to strategic importance to help our members work within their communities, and with state and federal partners to meet standards, manage risk, and be as secure and resilient as possible,” he said. “In 2017, 53 public power utilities participated in GridEx. This year we had 100. I think the surge in participation shows that public power utilities are taking these threats seriously and standing up to the challenge of facing them.”

The Association worked closely with NERC’s Electricity Information Sharing and Analysis Center (E-ISAC) for GridEx V to have more focus on impacts to distribution utilities, noted Sam Rozenberg, Engineering Services Security Director at the Association, who along with the public power utility subject matter experts as well as representatives from the rural electric cooperatives, led the increased engagement efforts. “Coupling that with increased awareness of cyber threats, resulted in the increase,” he said.

Robb said that the distributed play part of the exercise will end on Nov. 14. Separately, an executive tabletop exercise part of GridEx V will take place on Nov. 14 and involve various key government and industry leaders.

“As part of our dynamic approach, the tabletop has been adapted this year to focus on extraordinary operational measures that would be needed to restore the grid following a severe combined cyber and physical attack against our critical infrastructure,” he said.

“In this incarnation, we’re going to focus less on major policy issues and much more on real operational response and coordination that would be required to recover from such a devastating regional attack,” Robb said.

He said the scenario “that we’re using this year for the tabletop portion “is a focused regional attack in the northeastern part of North America and we chose that because it has a lot of very interesting characteristics. One is it highlights the dependency of natural gas, the natural gas interconnectivity between the two systems given the prevalence of natural gas in the generation mix in that area.”

Moreover, the scenario “brings in the financial services sector in spades because of the impacts that would occur to Wall Street and it also highlights the international collaboration and coordination that would need to be developed.”

Tabletop participants included industry executives from electricity, natural gas, telecommunications and finance, as well as officials from the Electricity Subsector Coordinating Council (ESCC), senior U.S. government officials, “as well as a number of our Canadian partners,” Robb said.

Other participants on the call with reporters were: Karen Evans, assistant secretary, Office of Cybersecurity, Energy Security and Emergency Response, Department of Energy; Brian Harrell, assistant director, Cybersecurity and Infrastructure Security Agency (CISA), Department of Homeland Security; Tom Fanning, ESCC co-chair, president and CEO, Southern Company; and Wailes, who along with his role at LES also serves as an ESCC co-chair.

“NERC’s grid security exercise is an opportunity for utilities to demonstrate how they would respond to and recover from a simulated and coordinated cyber and physical security attack,” Harrell said. “It’s also an opportunity to strengthen our crisis communications relationships with each other and then provide those lessons learned for industry to ultimately get better.”

For his part, Wailes compared the first GridEx to the fifth iteration of the exercise.

He noted that in 2011, there were very few security clearances in the industry. “We may not think we have enough but we’re working with our partners to make sure that we have a lot more clearances, a lot more information.”

In addition, Wailes pointed out that through the exercises, “we determined that we needed to develop” a cyber mutual aid program, “which basically emulates what we do during hurricanes as an industry in providing support,” recognizing that any one utility “may not have enough bench strength in a catastrophic event. We need to find ways to make that happen.”

Moreover, Wailes noted that there is “significantly more communication.” There are a variety of things that have been developed as a result of the exercises “that were the gaps that we discovered.”  

At a later point, Wailes said that “there is really no comparison” between the first GridEx and GridEx V.

“Most of us were strangers to each other on the stage with respect to the industry, the government representation. It was a cyber-only event. Obviously, it expanded to be cyber and physical over the years and the complexity got much higher,” Wailes said.  

Harrell said that in 2011, “I think there was a real apprehension to want to play in a cyber security exercise with your regulator. But I think NERC over the years has gotten well past that. They have demonstrated that there’s a trust relationship that goes well beyond compliance and the focus of the exercise is to collectively as an industry get better.”

Moreover, there has been an evolution of the scenario from one that was “very Stuxnet focused back in 2011 to now it’s a very hybrid threat landscape that we’re seeing and so many of the injects that we see include both a physical and cyber security nexus,” Harrell said.  

In terms of next steps, Robb noted that starting Nov. 15, “we’ll start with our first hot wash call on lessons learned. We will work with all the participants over the first quarter of 2020” and have an after-action report published early next year.

The American Public Power Association has developed the Public Power Cyber Incident Response Playbook to help community-owned utilities create a response plan, share information effectively, and identify who to engage during a cyber incident. The playbook would have been used by utilities during GridEx. Additional details about the playbook are available here.

NERC in April 2018 released a report that included lessons learned and recommendations in response to GridEx IV.

Among other things, the report said that the need for Cyber Mutual Assistance (CMA) was highlighted in 2015 following the executive tabletop portion of the NERC GridEx III exercise and was underscored by the December 2015 cyberattack in Ukraine.

The CMA program was developed through an industry-wide collaborative process and officially launched as a program of the ESCC in 2016. During GridEx IV, 42 electric companies participated in the CMA program as a reaction to the exercise scenario.

The CMA program provides a pool of utility cyber security experts who volunteer to share their expertise with other utilities in the event of a disruption of electric or natural gas service, systems, and/or IT infrastructure due to a cyber emergency. 

The Association encourages utilities to sign up for CMA. For more information, contact the Association Cybersecurity Team at: [email protected].