Skip to main content
SpringerLink
Log in

Q-learning-based sequential recovery of interdependent power-communication network after cascading failures

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Enhancing robustness of cyber-physical system under cascading failures still remains to be an important problem. In this paper, we propose a novel coupled model with the process of cascading failure, the DC power flow model and sequential bus/branch recovery in the interdependent power-communication network. Then, we adopt the Q-learning algorithm to search out the optimal recovery sequence with the minimal number of recovery times. By comparing the recovery costs of sequential bus recovery and sequential branch recovery, it is found that the strategy of branch recovery requires less recovery cost. Besides, we have also compared the performance of Q-learning-based branch recovery strategy with those of several other topology-related branch recovery strategies. Experimental results show that the Q-learning-based algorithm can effectively find the optimal sequence of branch recovery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Gao J, Buldyrev SV, Stanley HE, Havlin S (2012) Networks formed from interdependent networks. Nat Phys 8(1):40–48. https://doi.org/10.1038/nphys2180

    Article  Google Scholar 

  2. Buldyrev SV, Parshani R, Paul G, Stanley HE, Havlin S (2010) Catastrophic cascade of failures in interdependent networks. Nature 464(7291):1025–1028. https://doi.org/10.1038/nature08932

    Article  Google Scholar 

  3. Chen Z, Wu J, Xia Y, Zhang X (2017) Robustness of interdependent power grids and communication networks: A complex network perspective. IEEE Trans Circuits Syst II: Expr Briefs 65(1):115–119. https://doi.org/10.1109/tcsii.2017.2705758

    Article  Google Scholar 

  4. Li F, Yan X, Xie Y, Sang Z, Yuan X (2019) A review of cyber-attack methods in cyber-physical power system.2019 IEEE 8th International Conference on Advanced Power System Automation and Protection (APAP),pp 1335–1339. https://doi.org/10.1109/apap47170.2019.9225126

  5. Rueda DF, Calle E (2017) Using interdependency matrices to mitigate targeted attacks on interdependent networks: A case study involving a power grid and backbone telecommunications networks. Int J Crit Infrastruct Protection 16:3–12. https://doi.org/10.1016/j.ijcip.2016.11.004

    Article  Google Scholar 

  6. Shao J, Buldyrev SV, Havlin S, Stanley HE (2011) Cascade of failures in coupled network systems with multiple support-dependence relations. Phys Rev E 83(3):036116. https://doi.org/10.1103/physreve.83.036116

    Article  MathSciNet  Google Scholar 

  7. Huang Z, Wang C, Ruj Ruj, Stojmenovic M, Nayak A (2013) Modeling cascading failures in smart power grid using interdependent complex networks and percolation theory.2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA), pp 1023–1028. https://doi.org/10.1109/iciea.2013.6566517

  8. Ji X, Wang B, Liu D, Dong Z, Chen G, Zhu Z, Zhu X, Wang X (2016) Will electrical cyber-physical interdependent networks undergo first-order transition under random attacks? Phys Stat Mech Appl 460:235–245. https://doi.org/10.1016/j.physa.2016.05.017

    Article  MathSciNet  MATH  Google Scholar 

  9. Sun W, Liu C-C, Zhang L (2010) Optimal generator start-up strategy for bulk power system restoration. IEEE Trans Power Syst 26(3):1357–1366. https://doi.org/10.1109/tpwrs.2010.2089646

    Article  Google Scholar 

  10. Sarmadi SAN, Dobakhshari AS, Azizi S, Ranjbar AM (2011) A sectionalizing method in power system restoration based on wams. IEEE Trans Smart Grid 2(1):190–197. https://doi.org/10.1109/tsg.2011.2105510

    Article  Google Scholar 

  11. Wu J, Fang B, Fang J, Chen X, Chi KT (2019) Sequential topology recovery of complex power systems based on reinforcement learning. Phys Stat Mech Appl 535:122487. https://doi.org/10.1016/j.physa.2019.122487

    Article  Google Scholar 

  12. Zhang Y, Wu J, Chen Z, Huang Y, Zheng Z (2019) Sequential node/link recovery strategy of power grids based on q-learning approach. 2019 IEEE International Symposium on Circuits and Systems(ISCAS), pp 1–5. https://doi.org/10.1109/iscas.2019.8702107

  13. Jia H, Zheng H, Gai Y, Xu D (2019) Network recovery for large-scale failures in smart grid by simulation. International Conference on Simulation Tools and Techniques:477–485. https://doi.org/10.1109/compcomm.2018.8780720

  14. Huang Y, Wu J, Ren W, Chi KT, Zheng Z (2018) Sequential restorations of complex networks after cascading failures. IEEE Trans Syst, Man, Cybern: Syst 51(1):400–411. https://doi.org/10.1109/tsmc.2018.2874822

    Article  Google Scholar 

  15. Gong M, Ma L, Cai Q, Jiao L (2015) Enhancing robustness of coupled networks under targeted recoveries. Sci Rep 5(1):1–7. https://doi.org/10.1038/srep08439

    Article  Google Scholar 

  16. Li J, Li Y, Su Q (2022) Sequential recovery of cyber-physical power systems based on improved q-learning. Journal of the Franklin Institute(accepted). https://doi.org/10.1016/j.jfranklin.2022.05.043

  17. Hines P, Cotilla-Sanchez E, Blumsack S (2010) Do topological models provide good information about vulnerability in electric power networks?PubMed. https://doi.org/10.1063/1.3489887

  18. Gao X, Peng M, Chi KT, Zhan H (2020) A stochastic model of cascading failure dynamics in cyber-physical power systems. IEEE Syst J 14(3):4626–4637. https://doi.org/10.1109/jsyst.2020.2964624

    Article  Google Scholar 

  19. Cai Y, Cao Y, Li Y, Huang T, Zhou B (2015) Cascading failure analysis considering interaction between power grids and communication networks. IEEE Trans Smart Grid 7(1):530–538. https://doi.org/10.1109/tsg.2015.2478888

    Article  Google Scholar 

  20. Sydney A, Scoglio C, Youssef M, Schumm P (2010) Characterising the robustness of complex networks. Int J Internet Technol Secur Trans 2(3–4):291–320. https://doi.org/10.1504/ijitst.2010.037406

    Article  Google Scholar 

  21. Borgatti SP, Everett MG (2006) A graph-theoretic perspective on centrality. Soc Netw 28(4):466–484. https://doi.org/10.1016/j.socnet.2005.11.005

    Article  Google Scholar 

  22. Wu J, Chi KT, Lau FC (2015) Concept of node usage probability from complex networks and its applications to communication network design. IEEE Trans Circuits Syst I: Regular Papers 62(4):1195–1204. https://doi.org/10.1109/tcsi.2015.2399024

    Article  MathSciNet  MATH  Google Scholar 

  23. Sridhar S, Hahn A, Govindarasu M (2011) Cyber-physical system security for the electric power grid. Proc IEEE 100(1):210–224. https://doi.org/10.1049/pbpo081e_ch16

    Article  Google Scholar 

  24. Kim Y, Bae JN, Jin YK (2011) Performance of power line communication systems with noise reduction scheme for smart grid applications. IEEE Trans Consumer Electron 57(1):46–52. https://doi.org/10.1109/tce.2011.5735480

    Article  Google Scholar 

  25. Tang H, Lv K, Bak-Jensen B, Pillai JR, Wang Z (2022) Deep neural network-based hierarchical learning method for dispatch control of multiregional power grid. Neural Comput Appl 34(7):5063–5079. https://doi.org/10.1007/s00521-021-06008-4

    Article  Google Scholar 

  26. Wan Z, Li J, Gao Y (2018) Monitoring and diagnosis process of abnormal consumption on smart power grid. Neural Comput Appl 30(1):21–28. https://doi.org/10.1007/s00521-016-2719-4

    Article  Google Scholar 

  27. Villa D, Martin C, Villanueva FJ, Moya F, López JC (2011) A dynamically reconfigurable architecture for smart grids. IEEE Trans Consumer Electron 57(2):411–419. https://doi.org/10.1109/tce.2011.5955174

    Article  Google Scholar 

  28. Gupta S, Kambli R, Wagh S, Kazi F (2014) Support-vector-machine-based proactive cascade prediction in smart grid using probabilistic framework. IEEE Trans Industr Electron 62(4):2478–2486. https://doi.org/10.1109/tie.2014.2361493

    Article  Google Scholar 

  29. Motter AE (2004) Cascade control and defense in complex networks. Phys Rev Lett 93(9):098701. https://doi.org/10.1103/physrevlett.93.098701

    Article  MathSciNet  Google Scholar 

  30. Eppstein MJ, Hines PD (2012) A ‘random chemistry’ algorithm for identifying collections of multiple contingencies that initiate cascading failure. IEEE Trans Power Syst 27(3):1698–1705. https://doi.org/10.1109/pesmg.2013.6672899

    Article  Google Scholar 

  31. Stott B, Jardim J, Alsaç O (2009) Dc power flow revisited. IEEE Trans Power Syst 24(3):1290–1300. https://doi.org/10.1109/tpwrs.2009.2021235

    Article  Google Scholar 

  32. Huang W, Zhang T, Yao X (2022) Optimization for sequential communication line attack in interdependent power-communication network. Phys Stat Mech Appl 59:126837. https://doi.org/10.1016/j.physa.2021.126837

    Article  Google Scholar 

  33. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge. https://doi.org/10.1007/978-1-4615-3618-5_1

    Book  MATH  Google Scholar 

  34. Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3):279–292

    Article  MATH  Google Scholar 

  35. Chawla A, Agrawal P, Panigrahi BK, Paul K (2021) Deep-learningbased data-manipulation attack resilient supervisory backup protection of transmission lines. Neural Comput Appl 4:1–20. https://doi.org/10.1007/s00521-021-06106-3

    Article  Google Scholar 

  36. Li H, Wang L, Sun Z, Wang X (2019) Interdependent network cascading failure analysis based on load distribution and node backup strategy.2019 Chinese Control Conference (CCC), pp 5214–5219. https://doi.org/10.23919/chicc.2019.8866543

Download references

Acknowledgments

This work was supported by National Key R &D Program of China (2022YFE0198900), National Natural Science Foundation of China (61771430) and Basic Public Welfare Research Project of Zhejiang Province in China (LGG18F030002).

Author information

Author notes

  1. Yuxin Gao, Tianyi Zhang, Hua Gao have contributed equally to this work.

Authors and Affiliations

  1. Zhejiang University of Technology, Hangzhou, Zhejiang, 310014, China

    Wei Huang, Yuxin Gao, Tianyi Zhang & Hua Gao

Authors
  1. Wei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuxin Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tianyi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hua Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hua Gao.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, W., Gao, Y., Zhang, T. et al. Q-learning-based sequential recovery of interdependent power-communication network after cascading failures. Neural Comput & Applic 35, 12833–12845 (2023). https://doi.org/10.1007/s00521-023-08399-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08399-y

Keywords

Search

Navigation