A Regularized Tensor Completion Approach for PMU Data Recovery

Ghasemkhani, Amir; Niazazari, Iman; Liu, Yunchuan; Livani, Hanif; Centeno, Virgilio A.; Yang, Lei

doi:10.1109/tsg.2020.3030566

Title: A Regularized Tensor Completion Approach for PMU Data Recovery

Abstract

This article presents a novel data recovery framework for missing synchrophasor measurements. The imputation accuracy for the existing data recovery methods is significantly reduced when there are consecutive data losses across multiple data streams. Besides, the recovered data do not necessarily meet the physical constraints of the power grid. To tackle these issues, a regularized low rank tensor completion (LRTC) method is proposed to incorporate the domain knowledge (e.g., Kirchhoff's voltage and current laws, and three phase circuit relationships) as the regularization terms in order to efficiently exploit the data inter-dependencies for a better recovery. We leverage the tensor decomposition and completion as powerful tools to extract the latent structures of phasor measurement unit (PMU) data for the recovery process. Specifically, we first construct the tensor model of the PMU data and then formulate the LRTC problem as a rank minimization by leveraging the low rank property of the PMU measurements and adding the regularization terms into the LRTC problem in order to improve the imputation accuracy. An efficient algorithm based on alternating direction method of multipliers (ADMM) is developed to solve the regularized LRTC problem. Lastly, the experiments using the real PMU dataset show that the proposed approach exhibitsmore »« less

Authors:

Ghasemkhani, Amir ^[1]; Niazazari, Iman ^[2];

^[3]; Livani, Hanif ^[3]; Centeno, Virgilio A. ^[4];

^[3]

California State Univ., San Bernardino, CA (United States)
Electric Reliabilty Council of Texas, Taylor, TX (United States)
Univ. of Nevada, Reno, NV (United States)
Virginia Polytechnic Inst. and State Univ. (Virginia Tech), Blacksburg, VA (United States)

Publication Date:: Tue Oct 13 00:00:00 EDT 2020

Research Org.:: Univ. of Nevada, Reno, NV (United States)

Sponsoring Org.:: USDOE Office of Electricity (OE)

OSTI Identifier:: 1958802

Grant/Contract Number:: OE0000911

Resource Type:: Journal Article: Accepted Manuscript

Journal Name:: IEEE Transactions on Smart Grid

Additional Journal Information:: Journal Volume: 12; Journal Issue: 2; Journal ID: ISSN 1949-3053

Publisher:: IEEE

Country of Publication:: United States

Language:: English

Subject:: 42 ENGINEERING; missing data recovery; phasor measurement unit; tensor decomposition; regularized tensor completion

Citation Formats


                    Ghasemkhani, Amir, Niazazari, Iman, Liu, Yunchuan, Livani, Hanif, Centeno, Virgilio A., and Yang, Lei. A Regularized Tensor Completion Approach for PMU Data Recovery.  United States: N. p., 2020. 
        Web.  doi:10.1109/tsg.2020.3030566.

Copy to clipboard


                    Ghasemkhani, Amir, Niazazari, Iman, Liu, Yunchuan, Livani, Hanif, Centeno, Virgilio A., & Yang, Lei. A Regularized Tensor Completion Approach for PMU Data Recovery.  United States.  https://doi.org/10.1109/tsg.2020.3030566

Copy to clipboard


                    Ghasemkhani, Amir, Niazazari, Iman, Liu, Yunchuan, Livani, Hanif, Centeno, Virgilio A., and Yang, Lei. 2020.  
        "A Regularized Tensor Completion Approach for PMU Data Recovery".  United States.  https://doi.org/10.1109/tsg.2020.3030566.  https://www.osti.gov/servlets/purl/1958802.

Copy to clipboard


                    
@article{osti_1958802,

  title        = {A Regularized Tensor Completion Approach for PMU Data Recovery},

  author       = {Ghasemkhani, Amir and Niazazari, Iman and Liu, Yunchuan and Livani, Hanif and Centeno, Virgilio A. and Yang, Lei},

  abstractNote = {This article presents a novel data recovery framework for missing synchrophasor measurements. The imputation accuracy for the existing data recovery methods is significantly reduced when there are consecutive data losses across multiple data streams. Besides, the recovered data do not necessarily meet the physical constraints of the power grid. To tackle these issues, a regularized low rank tensor completion (LRTC) method is proposed to incorporate the domain knowledge (e.g., Kirchhoff's voltage and current laws, and three phase circuit relationships) as the regularization terms in order to efficiently exploit the data inter-dependencies for a better recovery. We leverage the tensor decomposition and completion as powerful tools to extract the latent structures of phasor measurement unit (PMU) data for the recovery process. Specifically, we first construct the tensor model of the PMU data and then formulate the LRTC problem as a rank minimization by leveraging the low rank property of the PMU measurements and adding the regularization terms into the LRTC problem in order to improve the imputation accuracy. An efficient algorithm based on alternating direction method of multipliers (ADMM) is developed to solve the regularized LRTC problem. Lastly, the experiments using the real PMU dataset show that the proposed approach exhibits better imputation accuracy, compared with the conventional matrix completion methods.},

  doi          = {10.1109/tsg.2020.3030566},

  url          = {https://www.osti.gov/biblio/1958802},
  journal      = {IEEE Transactions on Smart Grid},

  issn         = {1949-3053},

  number       = 2,

  volume       = 12,

  place        = {United States},

  year         = {Tue Oct 13 00:00:00 EDT 2020},

  month        = {Tue Oct 13 00:00:00 EDT 2020}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/tsg.2020.3030566

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

The Expression of a Tensor or a Polyadic as a Sum of Products
journal, April 1927

Hitchcock, Frank L.
Journal of Mathematics and Physics, Vol. 6, Issue 1-4
https://doi.org/10.1002/sapm192761164

Missing Data Recovery by Exploiting Low-Dimensionality in Power System Synchrophasor Measurements
journal, March 2016

Gao, Pengzhi; Wang, Meng; Ghiocel, Scott G.
IEEE Transactions on Power Systems, Vol. 31, Issue 2
https://doi.org/10.1109/TPWRS.2015.2413935

A Low-Rank Matrix Approach for the Analysis of Large Amounts of Power System Synchrophasor Data
conference, January 2015

Wang, Meng; Chow, Joe H.; Gao, Pengzhi
2015 48th Hawaii International Conference on System Sciences
https://doi.org/10.1109/HICSS.2015.318

Modeless reconstruction of missing synchrophasor measurements
conference, July 2014

Gao, Pengzhi; Wang, Meng; Ghiocel, Scott G.
2014 IEEE PES General Meeting: Conference & Exposition, 2014 IEEE PES General Meeting | Conference & Exposition
https://doi.org/10.1109/PESGM.2014.6938973

Estimate the Lost Phasor Measurement Unit Data Using Alternating Direction Multipliers Method
conference, April 2018

Liao, Mang; Shi, Di; Yu, Zhe
2018 IEEE/PES Transmission and Distribution Conference and Exposition (T&D)
https://doi.org/10.1109/TDC.2018.8440469

Modelless Data Quality Improvement of Streaming Synchrophasor Measurements by Exploiting the Low-Rank Hankel Structure
journal, November 2018

Hao, Yingshuai; Wang, Meng; Chow, Joe H.
IEEE Transactions on Power Systems, Vol. 33, Issue 6
https://doi.org/10.1109/TPWRS.2018.2850708

Multi-Channel missing data recovery by exploiting the low-rank hankel structures
conference, December 2017

Zhang, Shuai; Hao, Yingshuai; Wang, Meng
2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP)
https://doi.org/10.1109/CAMSAP.2017.8313138

A Singular Value Thresholding Algorithm for Matrix Completion
journal, January 2010

Cai, Jian-Feng; Candès, Emmanuel J.; Shen, Zuowei
SIAM Journal on Optimization, Vol. 20, Issue 4
https://doi.org/10.1137/080738970

Tensor Versus Matrix Completion: A Comparison With Application to Spectral Data
journal, July 2011

Signoretto, Marco; Van de Plas, Raf; De Moor, Bart
IEEE Signal Processing Letters, Vol. 18, Issue 7
https://doi.org/10.1109/LSP.2011.2151856

Short-Term Traffic Prediction Based on Dynamic Tensor Completion
journal, August 2016

Tan, Huachun; Wu, Yuankai; Shen, Bin
IEEE Transactions on Intelligent Transportation Systems, Vol. 17, Issue 8
https://doi.org/10.1109/TITS.2015.2513411

Tensor rank is NP-complete
journal, December 1990

Håstad, Johan
Journal of Algorithms, Vol. 11, Issue 4
https://doi.org/10.1016/0196-6774(90)90014-6

Wide-Area Frequency Monitoring Network (FNET) Architecture and Applications
journal, September 2010

Zhang, Yingchen; Markham, Penn; Xia, Tao
IEEE Transactions on Smart Grid, Vol. 1, Issue 2, p. 159-167
https://doi.org/10.1109/TSG.2010.2050345

Generating Unit Model Validation and Calibration Through Synchrophasor Measurements
journal, January 2015

Hajnoroozi, Ali A.; Aminifar, Farrokh; Ayoubzadeh, Hossein
IEEE Transactions on Smart Grid, Vol. 6, Issue 1
https://doi.org/10.1109/TSG.2014.2322821

Real Time Prediction and Control of Transient Stability Using Transient Energy Function
journal, January 2016

Bhui, Pratyasa; Senroy, Nilanjan
IEEE Transactions on Power Systems
https://doi.org/10.1109/TPWRS.2016.2564444

Tensor Decompositions and Applications
journal, August 2009

Kolda, Tamara G.; Bader, Brett W.
SIAM Review, Vol. 51, Issue 3
https://doi.org/10.1137/07070111X

Design, Testing, and Implementation of a Linear State Estimator in a Real Power System
journal, July 2017

Zhang, Lin; Bose, Anjan; Jampala, Anil
IEEE Transactions on Smart Grid, Vol. 8, Issue 4
https://doi.org/10.1109/TSG.2015.2508283

Design Requirements of Wide-Area Damping Systems—Using Empirical Data From a Utility IP Network
journal, March 2014

Zhu, Kun; Chenine, Moustafa; Nordstrom, Lars
IEEE Transactions on Smart Grid, Vol. 5, Issue 2
https://doi.org/10.1109/TSG.2013.2290493

Dimensionality Reduction of Synchrophasor Data for Early Event Detection: Linearized Analysis
journal, November 2014

Xie, Le; Chen, Yang; Kumar, P. R.
IEEE Transactions on Power Systems, Vol. 29, Issue 6
https://doi.org/10.1109/TPWRS.2014.2316476

Contingency-Constrained PMU Placement in Power Networks
journal, February 2010

Aminifar, F.; Khodaei, A.; Fotuhi-Firuzabad, M.
IEEE Transactions on Power Systems, Vol. 25, Issue 1
https://doi.org/10.1109/TPWRS.2009.2036470

An Alternating Direction Method of Multipliers Based Approach for PMU Data Recovery
journal, July 2019

Liao, Mang; Shi, Di; Yu, Zhe
IEEE Transactions on Smart Grid, Vol. 10, Issue 4
https://doi.org/10.1109/TSG.2018.2864176

Synchrophasor Measurement Technology in Power Systems: Panorama and State-of-the-Art
journal, January 2014

Aminifar, Farrokh; Fotuhi-Firuzabad, Mahmud; Safdarian, Amir
IEEE Access, Vol. 2
https://doi.org/10.1109/ACCESS.2015.2389659

Tensor Completion for Estimating Missing Values in Visual Data
journal, January 2013

Liu, Ji; Musialski, Przemyslaw; Wonka, Peter
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, Issue 1
https://doi.org/10.1109/TPAMI.2012.39

Accurate Recovery of Internet Traffic Data: A Sequential Tensor Completion Approach
journal, April 2018

Xie, Kun; Wang, Lele; Wang, Xin
IEEE/ACM Transactions on Networking, Vol. 26, Issue 2
https://doi.org/10.1109/TNET.2018.2797094

Smooth PARAFAC Decomposition for Tensor Completion
journal, October 2016

Yokota, Tatsuya; Zhao, Qibin; Cichocki, Andrzej
IEEE Transactions on Signal Processing, Vol. 64, Issue 20
https://doi.org/10.1109/TSP.2016.2586759

Tensor Completion Algorithms in Big Data Analytics
journal, January 2019

Song, Qingquan; Ge, Hancheng; Caverlee, James
ACM Transactions on Knowledge Discovery from Data, Vol. 13, Issue 1
https://doi.org/10.1145/3278607

Trace Norm Regularized CANDECOMP/PARAFAC Decomposition With Missing Data
journal, November 2015

Liu, Yuanyuan; Shang, Fanhua; Jiao, Licheng
IEEE Transactions on Cybernetics, Vol. 45, Issue 11
https://doi.org/10.1109/TCYB.2014.2374695

PMU Missing Data Recovery Using Tensor Decomposition
journal, November 2020

Osipov, Denis; Chow, Joe H.
IEEE Transactions on Power Systems, Vol. 35, Issue 6
https://doi.org/10.1109/TPWRS.2020.2991886

Similar records in OSTI.GOV collections:

Low-rank Tensor Completion for PMU Data Recovery

Conference Ghasemkhani, Amir; Liu, Yunchuan; Yang, Lei - 2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT)

This paper proposes a tensor completion method for the recovery of missing phasor measurement unit (PMU) measurements. Tensor completion as the general case of matrix completion has attracted increasing attention in recent years. The imputation accuracy for the existing matrix completion methods may be significantly reduced when there are consecutive data losses across multiple data channels. To tackle this issue, we explore the multi-way characteristics of PMU measurements by using a tensor model. We leverage the low-rank property of the PMU measurements and formulate the missing PMU data recovery problem as a low-rank tensor completion problem. An efficient algorithm basedmore »« less
https://doi.org/10.1109/isgt49243.2021.9372250

Full Text Available
Real-time Event Detection Using Rank Signatures of Real-world PMU Data

Conference Ghasemkhani, Amir; Liu, Yunchuan; Yang, Lei - 2022 IEEE Power & Energy Society General Meeting (PESGM)

Timely detection of power system events is a crucial task, which can facilitate the implementation of remedial actions to improve reliability, resiliency, and security of the system. Meanwhile, the widespread deployment of phasor measurement units (PMUs) makes it possible to develop data-driven event detection techniques. However, relying purely on data without incorporating domain knowledge for the event detection task in power systems poses substantial security and stability risks due to issues associated with data misinterpretation and model accuracy. In this regard, we propose a real-time event detection method using real-world PMU data by incorporating domain knowledge to adequately capture themore »« less
https://doi.org/10.1109/pesgm48719.2022.9917156

Full Text Available
Data Analytics for Electrical Distribution Systems with Micro PMUs (GMLC 1.4.9 Technical Report)

Technical Report Borges Hink, Raymond; Piesciorovsky, Emilio

This report describes how electrical utilities with residential, industrial, commercial and government customers make decisions using Data Analytics (DA) applications and discusses how OpenFMB was leveraged to share that data among disparate types of devices for DA. OpenFMB is a data standard developed by utilities for distributed communication and control of power system assets. Here we explored how OpenFMB can be used for sharing analytics and what those analytics could be for a set of representative grid devices. A review of DA applications performed by electrical utilities and customers is presented. An overview of how the testbed we used wasmore »« less
https://doi.org/10.2172/1649516

Full Text Available
Phasor-Measurement-Unit-Based Data Analytics Using Digital Twin and PhasorAnalytics Software

Technical Report Hart, Philip; Yan, Weizhong; Wang, Tianyi; ...

A major objective of this project was to apply GE’s commercial machine learning and data analytics toolsets to large-scale, real-world, anonymized Phasor Measurement Unit (PMU) datasets in order to extract signatures, correlated and/or causal factors, and precursor patterns associated with significant power system phenomena. The project had a particular emphasis on extraction of insights relevant to asset health monitoring, real-time load modeling and cybersecurity monitoring. Additionally, the team was directed to undertake a comprehensive data quality analysis for the provided datasets and encouraged to estimate the ‘machine-learning readiness’ of the datasets by documenting any major obstacles to the application ofmore »« less
https://doi.org/10.2172/1828164

Full Text Available
Weakly Supervised Event Classification Using Imperfect Real-world PMU Data with Scarce Labels

Conference Liu, Yunchuan; Yang, Lei - 2022 IEEE Power & Energy Society General Meeting (PESGM)

This paper studies event classification using imperfect real-world phasor measurement unit (PMU) data with scarce event types (labels). By investigating the real-world PMU data, it is observed that most real-world PMU data's event type is unknown, which makes it challenging to directly use such dataset to build event classifiers as existing classification techniques require high-quality training data with known event type (i.e., label). To address this challenge, a weakly supervised learning based event classification approach is developed, which can use noisy and low-quality PMU data for the training. First, data quality issues are fixed using data preprocessing techniques and thenmore »« less
https://doi.org/10.1109/pesgm48719.2022.9917176

Full Text Available

Similar Records

Title: A Regularized Tensor Completion Approach for PMU Data Recovery

Abstract

Citation Formats

The Expression of a Tensor or a Polyadic as a Sum of Products journal, April 1927

Missing Data Recovery by Exploiting Low-Dimensionality in Power System Synchrophasor Measurements journal, March 2016

A Low-Rank Matrix Approach for the Analysis of Large Amounts of Power System Synchrophasor Data conference, January 2015

Modeless reconstruction of missing synchrophasor measurements conference, July 2014

Estimate the Lost Phasor Measurement Unit Data Using Alternating Direction Multipliers Method conference, April 2018

Modelless Data Quality Improvement of Streaming Synchrophasor Measurements by Exploiting the Low-Rank Hankel Structure journal, November 2018

Multi-Channel missing data recovery by exploiting the low-rank hankel structures conference, December 2017

A Singular Value Thresholding Algorithm for Matrix Completion journal, January 2010

Tensor Versus Matrix Completion: A Comparison With Application to Spectral Data journal, July 2011

Short-Term Traffic Prediction Based on Dynamic Tensor Completion journal, August 2016

Tensor rank is NP-complete journal, December 1990

Wide-Area Frequency Monitoring Network (FNET) Architecture and Applications journal, September 2010

Generating Unit Model Validation and Calibration Through Synchrophasor Measurements journal, January 2015

Real Time Prediction and Control of Transient Stability Using Transient Energy Function journal, January 2016

Tensor Decompositions and Applications journal, August 2009

Design, Testing, and Implementation of a Linear State Estimator in a Real Power System journal, July 2017

Design Requirements of Wide-Area Damping Systems—Using Empirical Data From a Utility IP Network journal, March 2014

Dimensionality Reduction of Synchrophasor Data for Early Event Detection: Linearized Analysis journal, November 2014

Contingency-Constrained PMU Placement in Power Networks journal, February 2010

An Alternating Direction Method of Multipliers Based Approach for PMU Data Recovery journal, July 2019

Synchrophasor Measurement Technology in Power Systems: Panorama and State-of-the-Art journal, January 2014

Tensor Completion for Estimating Missing Values in Visual Data journal, January 2013

Accurate Recovery of Internet Traffic Data: A Sequential Tensor Completion Approach journal, April 2018

Smooth PARAFAC Decomposition for Tensor Completion journal, October 2016

Tensor Completion Algorithms in Big Data Analytics journal, January 2019

Trace Norm Regularized CANDECOMP/PARAFAC Decomposition With Missing Data journal, November 2015

PMU Missing Data Recovery Using Tensor Decomposition journal, November 2020

The Expression of a Tensor or a Polyadic as a Sum of Products
journal, April 1927

Missing Data Recovery by Exploiting Low-Dimensionality in Power System Synchrophasor Measurements
journal, March 2016

A Low-Rank Matrix Approach for the Analysis of Large Amounts of Power System Synchrophasor Data
conference, January 2015

Modeless reconstruction of missing synchrophasor measurements
conference, July 2014

Estimate the Lost Phasor Measurement Unit Data Using Alternating Direction Multipliers Method
conference, April 2018

Modelless Data Quality Improvement of Streaming Synchrophasor Measurements by Exploiting the Low-Rank Hankel Structure
journal, November 2018

Multi-Channel missing data recovery by exploiting the low-rank hankel structures
conference, December 2017

A Singular Value Thresholding Algorithm for Matrix Completion
journal, January 2010

Tensor Versus Matrix Completion: A Comparison With Application to Spectral Data
journal, July 2011

Short-Term Traffic Prediction Based on Dynamic Tensor Completion
journal, August 2016

Tensor rank is NP-complete
journal, December 1990

Wide-Area Frequency Monitoring Network (FNET) Architecture and Applications
journal, September 2010

Generating Unit Model Validation and Calibration Through Synchrophasor Measurements
journal, January 2015

Real Time Prediction and Control of Transient Stability Using Transient Energy Function
journal, January 2016

Tensor Decompositions and Applications
journal, August 2009

Design, Testing, and Implementation of a Linear State Estimator in a Real Power System
journal, July 2017

Design Requirements of Wide-Area Damping Systems—Using Empirical Data From a Utility IP Network
journal, March 2014

Dimensionality Reduction of Synchrophasor Data for Early Event Detection: Linearized Analysis
journal, November 2014

Contingency-Constrained PMU Placement in Power Networks
journal, February 2010

An Alternating Direction Method of Multipliers Based Approach for PMU Data Recovery
journal, July 2019

Synchrophasor Measurement Technology in Power Systems: Panorama and State-of-the-Art
journal, January 2014

Tensor Completion for Estimating Missing Values in Visual Data
journal, January 2013

Accurate Recovery of Internet Traffic Data: A Sequential Tensor Completion Approach
journal, April 2018

Smooth PARAFAC Decomposition for Tensor Completion
journal, October 2016

Tensor Completion Algorithms in Big Data Analytics
journal, January 2019

Trace Norm Regularized CANDECOMP/PARAFAC Decomposition With Missing Data
journal, November 2015

PMU Missing Data Recovery Using Tensor Decomposition
journal, November 2020