The North American Electric Reliability Corporation recently released a report that includes lessons learned and recommendations in response to GridEx, an exercise designed for utilities to exercise their response and recovery to cyber and physical security threats in a simulated environment, which took place in November.
Details on GridEx
The exercise, which takes place every two years, allows utilities, government partners and other critical infrastructure participants to engage with local and regional first responders, exercise cross-sector impacts, improve unity of messages and communication, identify lessons learned and engage senior leadership.
The exercise began in 2011 and NERC hosts the GridEx series. The 2017 GridEx, which took place over two days (Nov. 15-16), marked the fourth such exercise.
More than 1,100 public power employees and 53 public power organizations participated in the 2017 GridEx exercise. In 2015, the total number of public power organizations participating in GridEx was 26.
The exercise includes direct engagement among senior federal government officials and leaders from the Electricity Subsector Coordinating Council (ESCC).
The ESCC serves as the principal liaison between the federal government and the electric power industry and is comprised of the CEOs that represent all segments of the industry, including investor-owned electric companies, electric cooperatives, and public power utilities in the U.S. and Canada.
Kevin Wailes, administrator and CEO of Lincoln Electric System, serves as co-chair of the ESCC, while Sue Kelly, president and CEO of the American Public Power Association, serves on the ESCC steering committee.
The participation by Wailes and Kelly in GridEx activities helped to ensure that public power’s voice was heard throughout the two-day event.
With 6,500 individuals and 450 organizations participating across industry, law enforcement, and government agencies, GridEx IV consisted of a two-day distributed play exercise and a separate executive tabletop on the second day, the report noted.
The six-hour executive tabletop portion took place on Nov. 16 and involved industry executives and senior government officials. The 42 participants included a cross-section of industry executives and senior officials from federal and state governments. The tabletop was facilitated as a structured discussion for industry and government to share the actions they would take and issues they would face in responding to the scenario.
Objectives of GridEx IV
The report noted that the objectives of GridEx IV were to do the following: (1) Exercise incident response plans (2) Expand local and regional response (3) Engage critical interdependencies (4) Improve communication (5) Gather lessons learned; and (6) Engage senior leadership.
“NERC and participating organizations succeeded in achieving these objectives,” the report said. Responding to the after-action survey, 42 percent of participants indicated the exercise met their expectations “very well” and 55 percent indicated “well” for a total positive response of 97 percent.
However, 22 percent of participants responded that GridEx IV did not offer an effective opportunity for electric utilities to exercise their external communications response plans with external organizations, such as law enforcement and state emergency managers. External communication concerns should be addressed in future security exercises, the report said.
In addition to the after-action survey, participating organizations were encouraged to identify their own lessons learned and share them with NERC. NERC used this input to develop observations and propose recommendations to help the electricity industry enhance the security and reliability of North America’s bulk power system.
Distributed play
The report includes a chapter that provides a summary of the input NERC received from participating organizations that submitted after action survey responses and lessons learned documents.
This section of the report also included several observations and related recommendations.
For example, increasing the proactiveness of lead planners was one of the observations listed in the report.
The report said that the rapid growth and participation in GridEx has evolved into operating in a largely decentralized manner instead of a “one-size, fits-all” distributed play. “While the distributed play approach has been successful in the past, this caused some communication issues to arise during GridEx IV.”
The report said that lead planners were responsible for developing a scenario that would be tailored to their organizations’ needs while operating within the overarching GridEx design. “While lead planners were successful overall, some were inadequately prepared for the exercise or did not reach out to other stakeholders (such as local law enforcement or ICS product vendors), perhaps believing that the E-ISAC [Electricity Information Sharing and Analysis Center] or GridEx planning team would provide exercise injects or bring in the necessary organizations.”
One of the recommendations in this area is for lead planners to take a more proactive role in bringing in their necessary players, namely other electricity entities they operate with regularly, local law enforcement, government agencies, and utility equipment vendors. The report said that increased messaging from the GridEx Working Group “is needed to encourage organization outreach and that the NERC exercise planning team is a resource but not responsible for making decisions for any organization.”
Another observation relates to increase cyber mutual assistance program participation. The report said that the need for Cyber Mutual Assistance (CMA) was highlighted in 2015 following the executive tabletop portion of the NERC GridEx III exercise, and was underscored by the December 2015 cyberattack in Ukraine.
The CMA program was developed through an industry-wide collaborative process and officially launched as a program of the ESCC in 2016. During GridEx IV, 42 electric companies participated in the CMA program as a reaction to the exercise scenario. The CMA program provides a pool of utility cyber security experts who volunteer to share their expertise with other utilities in the event of a disruption of electric or natural gas service, systems, and/or IT infrastructure due to a cyber emergency.
Recommendations in this area are:
- More utilities should participate in the CMA program;
- The ESCC and the E-ISAC should engage more with the CMA program to efficiently share to encourage the efficient sharing of relevant information in the event, or in advance, of a cyber emergency in a manner that complies with the CMA program’s non-disclosure agreement; and
- The CMA program should continue to engage with potential partners and external stakeholders to discuss how best to be prepared to communicate about and respond to cyber emergencies when they arise.
(The Association encourages utilities to sign up for CMA. For more information, contact the Association Cybersecurity Team at: [email protected]).
Executive tabletop
Meanwhile, the report said that the tabletop reinforced the need to continue building on the collaborative relationships between the electricity industry and government. Participants recognized the progress made in a number of areas to address the recommendations of the 2013 GridEx II and 2015 GridEx III executive tabletops. They also acknowledged the need to continue efforts in other more challenging areas, such as unity of message and effort.
A recommendation in this area is for the ESCC and the Energy Government Coordinating Council to review and prioritize the recommendations in the report, assign ownership, decide how best to act on each of the recommendations, and provide periodic status updates to monitor progress in preparation for GridEx V in November 2019.
The report noted that participants highlighted two recommendations for particular attention from the perspective of maintaining reliable grid operations: (1) Increase Grid Emergency Response Capabilities: Address the need for emergency communications capabilities during severe events and (2) Ensure Utilities have Access to Sensitive Information: Quickly recognize threats that may affect multiple critical infrastructure sectors.
The report is available here.
