skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Big Data Analysis of Massive PMU Datasets: A Data Platform Perspective

Abstract

The discovery of `event signatures' and useful insights from very large historical Phasor Measurement Unit (PMU) datasets is predicated on offline Big Data analysis approaches that rely on the generation of predictive features on a massive scale. This paper presents lessons learned from a data platform perspective towards reducing barriers to adoption of Big Data analytics against a real dataset of almost half a trillion data points drawn from over 400 PMUs distributed across the North American power grid. We demonstrate software abstractions and targeted performance optimizations that can lead to significant productivity gains for power systems researchers seeking to perform offline exploratory temporal analysis and modeling tasks, with a focus on feature generation. We describe how our optimized approach goes beyond a naive application of mainstream Big Data technologies, enabling feature generation tasks, that previously took days or even weeks, to now be completed in just a few hours.

Authors:
; ; ; ; ;
Publication Date:
Research Org.:
GE Global Research
Sponsoring Org.:
USDOE
OSTI Identifier:
1971198
Report Number(s):
DOE-GE-0000915
DOE Contract Number:  
OE0000915
Resource Type:
Conference
Journal Name:
2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT)
Additional Journal Information:
Conference: 2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), Washington, DC, USA,16-18 February 2021
Country of Publication:
United States
Language:
English

Citation Formats

Kumar, Vijay S., Wang, Tianyi, Aggour, Kareem S., Wang, Pengyuan, Hart, Philip J., and Yan, Weizhong. Big Data Analysis of Massive PMU Datasets: A Data Platform Perspective. United States: N. p., 2021. Web. doi:10.1109/isgt49243.2021.9372203.
Kumar, Vijay S., Wang, Tianyi, Aggour, Kareem S., Wang, Pengyuan, Hart, Philip J., & Yan, Weizhong. Big Data Analysis of Massive PMU Datasets: A Data Platform Perspective. United States. https://doi.org/10.1109/isgt49243.2021.9372203
Kumar, Vijay S., Wang, Tianyi, Aggour, Kareem S., Wang, Pengyuan, Hart, Philip J., and Yan, Weizhong. 2021. "Big Data Analysis of Massive PMU Datasets: A Data Platform Perspective". United States. https://doi.org/10.1109/isgt49243.2021.9372203. https://www.osti.gov/servlets/purl/1971198.
@article{osti_1971198,
title = {Big Data Analysis of Massive PMU Datasets: A Data Platform Perspective},
author = {Kumar, Vijay S. and Wang, Tianyi and Aggour, Kareem S. and Wang, Pengyuan and Hart, Philip J. and Yan, Weizhong},
abstractNote = {The discovery of `event signatures' and useful insights from very large historical Phasor Measurement Unit (PMU) datasets is predicated on offline Big Data analysis approaches that rely on the generation of predictive features on a massive scale. This paper presents lessons learned from a data platform perspective towards reducing barriers to adoption of Big Data analytics against a real dataset of almost half a trillion data points drawn from over 400 PMUs distributed across the North American power grid. We demonstrate software abstractions and targeted performance optimizations that can lead to significant productivity gains for power systems researchers seeking to perform offline exploratory temporal analysis and modeling tasks, with a focus on feature generation. We describe how our optimized approach goes beyond a naive application of mainstream Big Data technologies, enabling feature generation tasks, that previously took days or even weeks, to now be completed in just a few hours.},
doi = {10.1109/isgt49243.2021.9372203},
url = {https://www.osti.gov/biblio/1971198}, journal = {2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT)},
number = ,
volume = ,
place = {United States},
year = {Tue Feb 16 00:00:00 EST 2021},
month = {Tue Feb 16 00:00:00 EST 2021}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:

Works referenced in this record:

A Hybrid Machine Learning Framework for Enhancing PMU-based Event Identification with Limited Labels
conference, May 2019


UPS: Unified PMU-Data Storage System to Enhance T+D PMU Data Usability
journal, January 2020


Distributed Data Analytics Platform for Wide-Area Synchrophasor Measurement Systems
journal, September 2016