Abstract
The complexity and NP-hard nature make finding optimal solutions challenging for combinatorial optimization problems (COPs) using traditional methods, especially for the large-scale problem. Recently, deep learning-based approaches have shown promise in solving COPs. Pointer Network (PN) has become a popular choice due to its ability to handle variable-length sequences and generate variable-sized outputs. Graph Pointer Network (GPN), which incorporates graph embedding layers in PN, can be well-suited for problems with graph structures. Additionally, Reinforcement Learning (RL) has great potential in enhancing scalability for solving large-scale instances. In this paper, we focus on Thinnest Path Problem (TPP). We propose an approach using RL to train GPN with constraints (GPN-c) to solve TPP. Our approach outperforms traditional solutions by providing faster and more efficient solving strategies. Specifically, we achieved significant improvements in solution quality, runtime, and scalability, and successfully extended our approach to instances with up to 500 nodes. Furthermore, RL and GPN can provide more flexible and adaptive solving strategies, making them highly applicable to real-world scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gao, J., Zhao, Q., Swami, A.: The thinnest path problem for secure communications. A directed hypergraph approach. In: Allerton Conference on Communication, Control, and Computing, pp. 847–852 (2012)
Moberly, R.: A thinner thinnest path using directional transmissions in a network. In: Military Communications Conference, pp. 1026–1031 (2013)
Gao, J., Zhao, Q., Swami, A.: The thinnest path problem. IEEE-ACM Trans. Networking 23(4), 1176–1189 (2015)
Wu, S., Chen, Z., Wang, Y., Gao, X., Wu, F., Chen, G.: Efficient approximations for thinnest path problem in grid graph. In: Ubiquitous Information Management and Communication, pp. 1–8 (2018)
Vinyals, O., Fortunato, M., Jaitly, N.: Pointer networks. In: Advances in Neural Information Processing Systems (2015)
Bello, I., Pham, H., Le, Q.V., Norouzi, M., Bengio, S.: Neural combinatorial optimization with reinforcement learning. In: International Conference on Learning Representations, pp. 1–8 (2017)
Nazari, M., Oroojlooy, A., Snyder, L., Takác, M.: Reinforcement learning for solving the vehicle routing problem. In: Advances in Neural Information Processing Systems (2018)
Dai, H., Khalil, E., Zhang, Y., Dilkina, B., Song, L.: Learning combinatorial optimization algorithms over graphs. In: Advances in Neural Information Processing Systems, pp. 6348–6358 (2017)
Ma, Q., Ge, S., He, D., Thaker, D., Drori, I.: Combinatorial optimization by graph pointer networks and hierarchical reinforcement learning. arXiv preprint arXiv:1911.04936 (2019)
Guo, T., Han, C., Tang, S., Ding, M.: Solving combinatorial problems with machine learning methods. In: Du, D.-Z., Pardalos, P.M., Zhang, Z. (eds.) Nonlinear Combinatorial Optimization. SOIA, vol. 147, pp. 207–229. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-16194-1_9
Kool, W., Hoof, H., Welling, M.: Attention, learn to solve routing problems!. In: International Conference on Learning Representations (2019)
Barrett, T., Clements, W., Foerster, J., Lvovsky, A.: Exploratory combinatorial optimization with reinforcement learning. In: AAAI Conference on Artificial Intelligence, vol. 34, pp. 3243–3250 (2020)
Joshi, C., Laurent, T., Bresson, X.: An efficient graph convolutional network technique for the travelling salesman problem. arXiv preprint arXiv:1906.01227 (2019)
Li, Z., Chen, Q., Koltun, V.: Combinatorial optimization with graph convolutional networks and guided tree search. In: Advances in Neural Information Processing Systems, pp. 539–548 (2018)
Bengio, Y., Lodi, A., Prouvost, A.: Machine learning for combinatorial optimization: a methodological tour d’horizon. Eur. J. Oper. Res. 290(2), 405–421 (2021)
Schuetz, M., Brubaker, J., Katzgraber, H.: Combinatorial optimization with physics-inspired graph neural networks. Nat. Mach. Intell. 4(4), 367–377 (2022)
Ma, H., Tu, S., Xu, L.: IA-CL: a deep bidirectional competitive learning method for traveling salesman problem. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds.) Neural Information Processing, ICONIP 2022. LNCS, vol. 13623, pp. 525–536. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-30105-6_44
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Gurobi Optimization, LLC: Gurobi optimizer reference manual (2022). Retrieved from http://www.gurobi.com
Dorigo, M.: Optimization, learning and natural algorithms. In: Thesis Politecnico Di Milano Italy (1992)
Acknowledgment
This work was supported in part by the Shanghai Artificial Intelligence Innovation and Development Fund grant 2020-RGZN-02026 and in part by the Shanghai Key Lab of Trustworthy Computing Chairman Fund 2022.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, J., Wang, Y., Zhang, C. (2024). Graph Pointer Network and Reinforcement Learning for Thinnest Path Problem. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1961. Springer, Singapore. https://doi.org/10.1007/978-981-99-8126-7_35
Download citation
DOI: https://doi.org/10.1007/978-981-99-8126-7_35
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8125-0
Online ISBN: 978-981-99-8126-7
eBook Packages: Computer ScienceComputer Science (R0)