Skip to main content
Springer Nature Link
Log in

Deep Reinforcement Learning Based on Greed for the Critical Cross-Section Identification Problem

  • Conference paper
  • First Online:
Data Science (ICPCSEE 2024)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2213))

  • 186 Accesses

Abstract

The critical cross-section identification problem (CCIP) presents a significant and highly challenging issue in power grid analysis, aiming to identify a partition of the graph into two disjoint cuts that maximize the total weight of the cut. Traditionally, critical cross-sections have been determined through manual experience or mechanistic analysis, and effective intelligent methods to address these issues are lacking. Therefore, we propose a deep reinforcement learning framework based on a greedy approach (DEER) to solve the CCIP problem. Initially, proven to be NP-hard, a greedy vertex merging approach is proposed that enables the acquisition of all CCIP solutions through vertex merging. To prevent the greedy algorithm from converging to local optima, a deep reinforcement learning (DRL) framework combined with vertex marking is proposed to simulate the Markov decision process of vertex merging. Through training the DRL model, repetitive searches for vertex marking can be effectively avoided. Furthermore, the greedy algorithm can be augmented with genetic algorithms to address CCIP. Extensive experiments demonstrate the effectiveness of the proposed methods in addressing CCIP.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhou, X., Zhang, Z.: Opinion maximization in social networks via leader selection[C]. In: Proceedings of the ACM Web Conference, pp. 133–142 (2023)

    Google Scholar 

  2. Zhou, H., Sun, F.: Topology path search method of active distribution network based on undirected graph[J]. Guangdong Electric Power 35(11), 64–71 (2022)

    Google Scholar 

  3. Eliasof, M., Haber, E., Treister, E.: Feature transportation improves graph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38 no. 11, pp. 11874–11882 (2024)

    Google Scholar 

  4. Grinsztajn, N., Furelos-Blanco, D., Surana, S., et al.: Winner takes it all: training performant RL populations for combinatorial optimization[J]. Adv. Neural Inform. Process. Syst. 36 (2024)

    Google Scholar 

  5. Voigt, B.F.: Der Handlungsreisende, wie er sein soll und was er zu thun hat, um Aufträge zu erhalten und eines glücklichen Erfolgs in seinen Geschäften gewiss zu sein[J], pp. 69–72. Ilmenau, Commis-Voageur (1831)

    Google Scholar 

  6. Garey, M.R., Johnson, D.S.: Computers and intractability[M]. San Francisco: freeman (1979)

    Google Scholar 

  7. Miller, R.E., Muller, D.E.: A problem of maximum consistent subsets[R]. IBM Research Report RC-240, JT Watson Research Center, Yorktown Heights, NY, (1960)

    Google Scholar 

  8. Wang, T., Li, Y., Gu, X., et al.: Identification of the key transmission sections considering optimization of geographical partition boundary for power grids [J]. Trans. China Electrotech. Society 29(04), 220–228+245 (2014)

    Google Scholar 

  9. Zhang, X., Grijalva, S.: Decentralized total transfer capability evaluation using domain decomposition methods[J]. IEEE Trans. Power Syst. 31(5), 3349–3357 (2015)

    Article  Google Scholar 

  10. Min, L., Abur, A.: Total transfer capability computation for multi-area power systems. IEEE Trans. Power Syst. 21(3), 1141–1147 (2006)

    Article  Google Scholar 

  11. Yan, Y., Zhou, Q., He, H., et al.: Subarea division and transmission sections search method based on complex network theory. Electric Power Construct. 38(6), 100–107 (2017)

    Google Scholar 

  12. Papadimitriou, C., Steiglitz, K.: Combinatorial optimization: algorithms and complexity[M]. Courier Corporation (1998)

    Google Scholar 

  13. Lawler, E., Wood, D.: Branch-and-bound methods: a survey. Oper. Res. 14(4), 699–719 (1966)

    Article  MathSciNet  Google Scholar 

  14. Bertsekas, D.P.: Dynamic Programming and Optimal Control. Athena Scientific, Belmont, MA (1995)

    Google Scholar 

  15. Sniedovich, M.: Dynamic Programming: Foundations and Principles (Second edition). CRC Press, Boca Raton, FL (2010)

    Book  Google Scholar 

  16. Williamson, D., Shmoys, D.: The Design of Approximation Algorithms. Cambridge University Press, Cambridge (2011)

    Book  Google Scholar 

  17. Vazirani, V.: Approximation Algorithms. Springer, Berlin, Heidelberg (2003)

    Book  Google Scholar 

  18. Hochba, D.: Approximation algorithms for NP-hard problems. ACM SIGACT News 28(2), 40–52 (1997)

    Article  Google Scholar 

  19. Chen, X., Tian, Y.: Learning to perform local rewriting for combinatorial optimization. In: Proceedings of the 33rd Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates, Inc., pp. 6278–6289 (2019)

    Google Scholar 

  20. Yolcu, E., Poczos, B.: Learning local search heuristics for Boolean satisfiability. In: Proceedings of the 33rd Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates, Inc., pp. 7992–8003 (2019)

    Google Scholar 

  21. Gao, L., Chen, M., Chen, Q., et al.: Learn to design the heuristics for vehicle routing problem. arXiv preprint arXiv: 2002.08539 (2020)

  22. Lu, H., Zhang, X., Yang, S.: A learning-based iterative method for solving vehicle routing problems. In: Proceedings of the 8th International Conference on Learning Representations. Addis Ababa, Ethiopia, (2020)

    Google Scholar 

  23. Karalias, N., Loukas, A.: Erdos goes neural: an unsupervised learning framework for combinatorial optimization on graphs. Adv. Neural. Inf. Process. Syst. 33, 6659–6672 (2020)

    Google Scholar 

  24. Barrett, T., Clements, W., Foerster, J., et al.: Exploratory combinatorial optimization with reinforcement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34 no. 04, pp. 3243–3250 (2020)

    Google Scholar 

  25. Wang, H., Wu, N., Yang, H., et al.: Unsupervised learning for combinatorial optimization with principled objective relaxation. Adv. Neural. Inf. Process. Syst. 35, 31444–31458 (2022)

    Google Scholar 

  26. Aarts, E., Aarts, E., Lenstra, J.: Local search in combinatorial optimization[M]. Princeton University Press (2003)

    Google Scholar 

  27. Sivanandam, S., Deepa, S.: Genetic Algorithms[M], pp. 15–37. Berlin, Heidelberg, Introduction to genetic algorithms. Springer (2008)

    Google Scholar 

  28. Dorigo, M., Birattari, M., Stutzle, T.: Ant colony optimization. IEEE Comput. Intell. Mag. 1(4), 28–39 (2006)

    Article  Google Scholar 

  29. Wu, C., Shankari, K., Kamar, E., et al.: Optimizing the diamond lane: a more tractable carpool problem and algorithms. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). IEEE, pp. 1389–1396 (2016)

    Google Scholar 

  30. Croes, G.: A method for solving traveling-salesman problems. Oper. Res. 6(6), 791–812 (1958)

    Article  MathSciNet  Google Scholar 

  31. Helsgaun, K.: An effective implementation of the Lin-Kernighan traveling salesman heuristic. Eur. J. Oper. Res. 126(1), 106–130 (2000)

    Article  MathSciNet  Google Scholar 

  32. Lin, S., Kernighan, B.: An effective heuristic algorithm for the traveling-salesman problem. Oper. Res. 21(2), 498–516 (1973)

    Article  MathSciNet  Google Scholar 

  33. Lin, S.: Computer solutions of the traveling salesman problem. Bell Labs Tech. J. 44(10), 2245–2269 (1965)

    Article  MathSciNet  Google Scholar 

  34. Helsgaun, K.: An Extension Of The Lin-kernighan-helsgaun Tsp Solver for Constrained Traveling Salesman nd Vehicle Routing Problems, pp. 24–50. Roskilde University, Roskilde (2017)

    Google Scholar 

  35. Vidal, T.: Hybrid genetic search for the CVRP: Open-source implementation and SWAP* neighborhood. Comput. Oper. Res. 140, 105643 (2022)

    Article  MathSciNet  Google Scholar 

  36. Vidal, T., Crainic, T., Gendreau, M., et al.: A hybrid genetic algorithm for multidepot and periodic vehicle routing problems. Oper. Res. 60(3), 611–624 (2012)

    Article  MathSciNet  Google Scholar 

  37. Bourel, M., Canale, E., Robledo, F., et al.: Complexity and heuristics for the weighted max cut-clique problem. Int. Trans. Oper. Res. 29(2), 908–928 (2022)

    Article  MathSciNet  Google Scholar 

  38. Nogueira, B., Pinheiro, R., Subramanian, A.: A hybrid iterated local search heuristic for the maximum weight independent set problem. Optimiz. Lett. 12(3), 567–583 (2018)

    Article  MathSciNet  Google Scholar 

  39. Vinyals, O., Fortunato, M., Jaitly, N.: vertexer networks. Adv. Neural Inform. Process. Systems, vol. 28 (2015)

    Google Scholar 

  40. Li, Z., Chen, Q., Koltun, V.: Combinatorial optimization with graph convolutional networks and guided tree search. Adv. Neural Inform. Process. Syst. vol. 31 (2018)

    Google Scholar 

  41. Selsam, D., Lamm, M., Bünz, B., et al.L Learning a SAT solver from single-bit supervision[C]. In: 7th International Conference on Learning Representations(ICLR 2019) (2019)

    Google Scholar 

  42. Bello, I., Pham, H., Le, Q., et al.: Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940 (2016)

  43. Khalil, E., Dai, H., Zhang, Y., et al.: Learning combinatorial optimization algorithms over graphs[J]. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  44. Deudon, M., Cournut, P., Lacoste, A., et al.: Learning heuristics for the tsp by policy gradient. In: International conference on the integration of constraint programming, artificial intelligence, and operations research. Springer, Cham, pp. 170–181 (2018). https://doi.org/10.1007/978-3-319-93031-2_12

  45. Kool, W., Van Hoof, H., Welling, M.: Attention, Learn to Solve Routing Problems!. In: 7th International Conference on Learning Representations (ICLR 2019) (2019)

    Google Scholar 

  46. Hopfield, J., Tank, D.: Neural computation of decisions in optimization problemsD. Biol. Cybern. 52(3), 141–152 (1985)

    Article  Google Scholar 

  47. Smith, K.: Neural networks for combinatorial optimization: a review of more than a decade of research. INFORMS J. Comput. 11(1), 15–34 (1999)

    Article  MathSciNet  Google Scholar 

  48. Amizadeh, S., Matusevych, S., Weimer, M.: Learning to solve circuit-SAT: an unsupervised differentiable approach. In: International Conference on Learning Representations (2018)

    Google Scholar 

  49. Álvarez-Miranda, E., Ljubić, I., Mutzel, P.: The Maximum Weight Connected Subgraph Problem, pp. 245–270. Festschrift for Martin Grötschel, Facets of combinatorial optimization (2013)

    Google Scholar 

  50. Wei, Q., Tang, W., Jiang, C., et al.: Online equivalent modeling of active distribution network based on improved reinforcement learning algorithm. Guangdong Electric Power 34(11), 19–26 (2021)

    Google Scholar 

Download references

Acknowledgments

This paper was supported by the Science and Technology Project of State Grid: Research on artificial intelligence analysis technology of available transmission capacity (ATC) of the key section under multiple power grid operation modes (5100-202255020A-1-1-ZN).

Author information

Authors and Affiliations

  1. Harbin Institute of Technology, Harbin, China

    Huaiyuan Liu, Donghua Yang, Hekai Huang & Hongzhi Wang

  2. China Electric Power Research Institute, Beijing, China

    Xinglei Chen

  3. State Grid Shanghai Municipal Electric Power Company, Shanghai, China

    Yong Cui & Jun Gu

Authors
  1. Huaiyuan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Donghua Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hekai Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinglei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongzhi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yong Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jun Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongzhi Wang .

Editor information

Editors and Affiliations

  1. University of Macau, Macau, China

    Chengzhong Xu

  2. Harbin Engineering University, Harbin, China

    Haiwei Pan

  3. Huazhong University of Science and Technology, Wuhan, China

    Chen Yu

  4. City University of Hong Kong, Kowloon Tong, China

    Jianping Wang

  5. Harbin Engineering University, Harbin, China

    Qilong Han

  6. Harbin University of Science and Technology, Harbin, China

    Xianhua Song

  7. National Academy of Guo Ding Institute of Data Science, Beijing, China

    Zeguang Lu

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, H. et al. (2024). Deep Reinforcement Learning Based on Greed for the Critical Cross-Section Identification Problem. In: Xu, C., et al. Data Science. ICPCSEE 2024. Communications in Computer and Information Science, vol 2213. Springer, Singapore. https://doi.org/10.1007/978-981-97-8743-2_9

Download citation

Publish with us

Policies and ethics