Powering Strong Communities
Electricity Markets

DOE Initiative Makes Energy Data Available Via "Data Lakes"

Like What You Are Reading?

Please take a few minutes to let us know what type of industry news and information is most meaningful to you, what topics you’re interested in, and how you prefer to access this information.

The Open Energy Data Initiative -- currently funded by the U.S. Department of Energy  Solar Energy Technologies Office and developed by the National Renewable Energy Laboratory -- makes energy data widely available to anyone by providing access to big data via "data lakes," large collections of open-access energy data.

“In the past, ‘big data’ -- data too large or too complex (or both) to be easily consumed by conventional means -- could only be used by people with access to high-performance computing systems or supercomputers. OEDI makes vast amounts of energy data available for schools, companies just getting started, and data enthusiasts of all stripes. Each dataset includes access instructions and examples of how to work with the data,” NREL noted.

Accessible from OpenEI, OEDI is a centralized location that can house data generated across the 17 national laboratories and all their partner organizations for research and development funded through DOE, with additional contributions from other federal agencies and private companies.

Currently, over 200 providers from government labs, private industries, and universities contribute to OEDI “and that data is available to anyone with a laptop and an internet connection to access and use in the cloud,” NREL said.

OEDI's data repository runs the gamut, spanning high-resolution data for atmospheric conditions used every day by U.S. weather forecasters; real-world building data in hourly increments that shows buildings' energy consumption to serve as examples for engineers and architects; high-resolution data that shows how much wind or solar energy a given location can produce, not only for the United States but also many regions around the globe; databases about drilling and laser drilling; raw materials and supply chain analysis; and beyond, in over 2,000 datasets.

"Limited access to big data historically excluded startups, small organizations, and minority-serving institutions from research collaborations and community initiatives," SETO Director Becca Jones-Albertus said. "OEDI gives universal access to big data, fostering inclusive partnerships that enable better decision-making throughout the clean energy transition."

NREL said that traditional data repositories "are not typically optimized for discovering or sharing data. Huge amounts of data further complicate data sharing by making the cost of storage and the time needed to transfer the data prohibitive, limiting access and reducing the potential for collaboration and innovation. Stored on the cloud, OEDI solves these issues by providing a centralized location for large energy datasets."

Making energy data free and widely accessible "allows communities to work with researchers and the energy industry to find solutions that meet the communities' needs, fostering innovation that benefits everyone."

OEDI further supports accessibility by making data easier to use, NREL said.

OEDI data pipelines automatically detect certain raw data formats and standardize them, displaying the standardized data right alongside the raw data for scientific posterity.

OEDI data standards adhere to the precedents from the U.S. General Services Administration's Open Data and FAIR (findable, accessible, interoperable, and reusable) data principles.

"With DOE support, NREL built OEDI from the ground up to share data," said Jon Weers, who leads OEDI for NREL. "Anyone can access the data through OEDI directly or through a network of data sharing partners like Data.gov, Google Datasets, and dozens of other sites."

NREL said that OEDI is already impacting communities. The OEDI team supported communities in Puerto Rico in their transition to renewable energy in response to hurricanes Maria and Fiona.

Working with DOE, the U.S. Census' The Opportunity Project, local governments, and local teams, OEDI researchers facilitated the development of five different innovative solutions to help Puerto Rico increase its resilience and transition to 100% renewable energy.

These solutions were developed by small teams of local collaborators focusing on community-driven solutions by applying and analyzing big data freely accessible through OEDI, which was recognized by Puerto Rico Governor Pedro Pierluisi during the TOP Summit.

 

NEW Topics