A Multiagent Deep Reinforcement Learning-Enabled Dual-Branch Damping Controller for Multimode Oscillation

Zhang, Guozhou; Zhao, Junbo; Hu, Weihao; Cao, Di; Kamwa, Innocent; Duan, Nan; Chen, Zhe

doi:10.1109/tcst.2022.3176736

Title: A Multiagent Deep Reinforcement Learning-Enabled Dual-Branch Damping Controller for Multimode Oscillation

Journal Article · Thu Jun 09 00:00:00 EDT 2022 · IEEE Transactions on Control Systems Technology

DOI:https://doi.org/10.1109/tcst.2022.3176736· OSTI ID:1893590

^[1];

^[2];

^[1]; Cao, Di ^[1];

^[3];

^[4];

^[5]

University of Electronic Science and Technology of China, Chengdu (China)
University of Connecticut, Storrs, CT (United States)
Laval University, Quebec City (Canada)
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
Aalborg University (Denmark)

Here, this study develops a multiagent deep reinforcement learning (MADRL)-enabled framework for the decentralized cooperative control of a novel dual-branch (DB) damping controller for both low-frequency oscillation (LFO) and ultralow-frequency oscillation (ULFO). It has two branches, each of which consists of a proportional resonance (PR) and a second-order polynomial that is designed to handle target oscillation modes. To improve the robustness of the controller to system uncertainties, MADRL is developed, where multiagents are centrally trained to obtain the coordinated adaptive control policy while being executed in a decentralized manner to provide the optimal parameter setting for each controller with only local states. Comparisons with the IEEE 10-machine 39-bus system demonstrate that the proposed method achieves better robustness to uncertainties, lower communication delay, and single-point failure, as well as damping control performances for both LFO and ULFO.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); National Key Research and Development Program of China

Grant/Contract Number:: AC52-07NA27344; 2018YFE0127600

OSTI ID:: 1893590

Report Number(s):: LLNL-JRNL-835974; 1055187

Journal Information:: IEEE Transactions on Control Systems Technology, Vol. N/A, Issue N/A; ISSN 1063-6536

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (25)

Multivariable Self-Tuning Feedback Linearization Controller for Power Oscillation Damping Arif, Jawad; Ray, Swakshar; Chaudhuri, Balarko IEEE Transactions on Control Systems Technology, Vol. 22, Issue 4 https://doi.org/10.1109/TCST.2013.2279939	journal	July 2014
A Probabilistic Robust Coordinated Approach to Stabilize Power Oscillations in DFIG-Based Power Systems Morshed, Mohammad Javad; Fekih, Afef IEEE Transactions on Industrial Informatics, Vol. 15, Issue 10 https://doi.org/10.1109/TII.2019.2901935	journal	October 2019
Feature Selection Method for Estimating Systolic Blood Pressure Using the Taguchi Method Suzuki, Arata; Ryu, Kazuteru IEEE Transactions on Industrial Informatics, Vol. 10, Issue 2 https://doi.org/10.1109/TII.2013.2288498	journal	May 2014
Deep Reinforcement Learning-Based Approach for Proportional Resonance Power System Stabilizer to Prevent Ultra-Low-Frequency Oscillations Zhang, Guozhou; Hu, Weihao; Cao, Di IEEE Transactions on Smart Grid, Vol. 11, Issue 6 https://doi.org/10.1109/TSG.2020.2997790	journal	November 2020
Online emergency control to suppress frequency oscillations based on damping evaluation using dissipation energy flow Chen, Lei; Min, Yong; Lu, Xiaomin International Journal of Electrical Power & Energy Systems, Vol. 103 https://doi.org/10.1016/j.ijepes.2018.06.022	journal	December 2018
Optimization Algorithms for Catching Data Manipulators in Power System Estimation Loops Liao, Mang; Chakrabortty, Aranya IEEE Transactions on Control Systems Technology, Vol. 27, Issue 3 https://doi.org/10.1109/TCST.2018.2805294	journal	May 2019
A Novel Adaptive Wide Area PSS Based on Output-Only Modal Analysis Zhang, Junbo; Chung, C. Y.; Lu, Chao IEEE Transactions on Power Systems, Vol. 30, Issue 5 https://doi.org/10.1109/TPWRS.2014.2362923	journal	September 2015
IEEE PSS2B Versus PSS4B: The Limits of Performance of Modern Power System Stabilizers Kamwa, I.; Grondin, R.; Trudel, G. IEEE Transactions on Power Systems, Vol. 20, Issue 2 https://doi.org/10.1109/TPWRS.2005.846197	journal	May 2005
Model‐based tuning approach for multi‐band power system stabilisers PSS4B using an improved modal performance index Rimorov, Dmitry; Kamwa, Innocent; Joós, Géza IET Generation, Transmission & Distribution, Vol. 9, Issue 15 https://doi.org/10.1049/iet-gtd.2014.1176	journal	November 2015
Online PMU-Based Wide-Area Damping Control for Multiple Inter-Area Modes Zenelis, Ilias; Wang, Xiaozhe; Kamwa, Innocent IEEE Transactions on Smart Grid, Vol. 11, Issue 6 https://doi.org/10.1109/TSG.2020.3001607	journal	November 2020
Learning Distributed Stabilizing Controllers for Multi-Agent Systems Jing, Gangshan; Bai, He; George, Jemin IEEE Control Systems Letters, Vol. 6 https://doi.org/10.1109/LCSYS.2021.3072007	journal	January 2022
Scalable Designs for Reinforcement Learning-Based Wide-Area Damping Control Mukherjee, Sayak; Chakrabortty, Aranya; Bai, He IEEE Transactions on Smart Grid, Vol. 12, Issue 3 https://doi.org/10.1109/TSG.2021.3050419	journal	May 2021
A novel deep reinforcement learning enabled sparsity promoting adaptive control method to improve the stability of power systems with wind energy penetration Zhang, Guozhou; Hu, Weihao; Cao, Di Renewable Energy, Vol. 178 https://doi.org/10.1016/j.renene.2021.06.081	journal	November 2021
Q-Learning-Based Damping Control of Wide-Area Power Systems Under Cyber Uncertainties Duan, Jiajun; Xu, Hao; Liu, Wenxin IEEE Transactions on Smart Grid, Vol. 9, Issue 6 https://doi.org/10.1109/TSG.2017.2711599	journal	November 2018
Gradient based hybrid metaheuristics for robust tuning of power system stabilizers Peres, Wesley; Silva Júnior, Ivo Chaves; Passos Filho, João Alberto International Journal of Electrical Power & Energy Systems, Vol. 95 https://doi.org/10.1016/j.ijepes.2017.08.014	journal	February 2018
Robust Adaptive Dynamic Programming for Large-Scale Systems With an Application to Multimachine Power Systems Jiang, Yu; Jiang, Zhong-Ping IEEE Transactions on Circuits and Systems II: Express Briefs, Vol. 59, Issue 10 https://doi.org/10.1109/TCSII.2012.2213353	journal	October 2012
Wide-Area Measurement System-Based Low Frequency Oscillation Damping Control Through Reinforcement Learning Hashmy, Yousuf; Yu, Zhe; Shi, Di IEEE Transactions on Smart Grid, Vol. 11, Issue 6 https://doi.org/10.1109/TSG.2020.3008364	journal	November 2020
Off-Policy Deep Reinforcement Learning Based on Steffensen Value Iteration Cheng, Yuhu; Chen, Lin; Chen, C. L. Philip IEEE Transactions on Cognitive and Developmental Systems, Vol. 13, Issue 4 https://doi.org/10.1109/TCDS.2020.3034452	journal	December 2021
Adaptive Signal Selection of Wide-Area Damping Controllers Under Various Operating Conditions Surinkaew, Tossaporn; Ngamroo, Issarachai IEEE Transactions on Industrial Informatics, Vol. 14, Issue 2 https://doi.org/10.1109/TII.2017.2752762	journal	February 2018
Validation of a new modal performance measure for flexible controllers design Simo, J. B.; Kamwa, I.; Trudel, G. IEEE Transactions on Power Systems, Vol. 11, Issue 2 https://doi.org/10.1109/59.496160	journal	May 1996
Optimization of Governor Parameters to Prevent Frequency Oscillations in Power Systems Chen, Lei; Lu, Xiaomin; Min, Yong IEEE Transactions on Power Systems, Vol. 33, Issue 4 https://doi.org/10.1109/TPWRS.2017.2778506	journal	July 2018
Optimization Strategy of Hydrogovernors for Eliminating Ultralow-Frequency Oscillations in Hydrodominant Power Systems Chen, Gang; Tang, Fan; Shi, Huabo IEEE Journal of Emerging and Selected Topics in Power Electronics, Vol. 6, Issue 3 https://doi.org/10.1109/JESTPE.2017.2788004	journal	September 2018
Fundamental study of common mode small-signal frequency oscillations in power systems Xie, Ruichao; Kamwa, Innocent; Rimorov, Dmitry International Journal of Electrical Power & Energy Systems, Vol. 106 https://doi.org/10.1016/j.ijepes.2018.09.042	journal	March 2019
Analysis and Measures of Ultralow-Frequency Oscillations in a Large-Scale Hydropower Transmission System Mo, Weike; Chen, Yiping; Chen, Haoyong IEEE Journal of Emerging and Selected Topics in Power Electronics, Vol. 6, Issue 3 https://doi.org/10.1109/JESTPE.2017.2759810	journal	September 2018
Analysis of Very Low Frequency Oscillations in Hydro-Dominant Power Systems Using Multi-Unit Modeling Villegas Pico, Hugo; McCalley, James D.; Angel, Andrea IEEE Transactions on Power Systems, Vol. 27, Issue 4 https://doi.org/10.1109/TPWRS.2012.2187805	journal	November 2012

Similar Records

Attention Enabled Multi-Agent DRL for Decentralized Volt-VAR Control of Active Distribution System Using PV Inverters and SVCs

Journal Article · Thu Feb 04 00:00:00 EST 2021 · IEEE Transactions on Sustainable Energy · OSTI ID:1893590

Cao, Di; Zhao, Junbo; Hu, Weihao; +3 more

Data-Driven Multi-agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control with High Penetration of PVs

Journal Article · Fri Apr 09 00:00:00 EDT 2021 · IEEE Transactions on Smart Grid · OSTI ID:1893590

Cao, Di; Zhao, Junbo; Hu, Weihao; +4 more

Coordinated Control of Natural and Sub-Synchronous Oscillations via HVDC Links in Great Britain Power System

Conference · Fri Apr 01 00:00:00 EDT 2022 · OSTI ID:1893590

Zhao, Yi; Dong, Yuqing; Zhu, Lin; +11 more

Related Subjects

97 MATHEMATICS AND COMPUTING
dual-branch damping controller
low-frequency oscillation
multiagent deep reinforcement learning
power system control
ultralow-frequency oscillation

Title: A Multiagent Deep Reinforcement Learning-Enabled Dual-Branch Damping Controller for Multimode Oscillation

Citation Formats

References (25)

Similar Records

Related Subjects