Skip to main content
Springer Nature Link
Log in

Solving an Instance of a Routing Problem Through Reinforcement Learning and High Performance Computing

  • Conference paper
  • First Online:
Cloud Computing, Big Data & Emerging Topics (JCC-BD&ET 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1634))

Included in the following conference series:

  • 431 Accesses

Abstract

Today, data management is important, mainly in organizations where the real-time processing of a large number of events is important to decision-making systems. Analyzing large amounts of data through analytics allows discovering hidden knowledge and make decisions in consequence. In this work, we propose to solve a decision-making problem in real-time using prescriptive analytics model, reinforcement learning agents and parallel computing techniques in GPU. Particularly, we consider the vehicle routing problem (VRP) with real-time information provision and re-routing. The experimental results confirm that the adequate combination of these techniques is a promising option for solving this kind of problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Abadi, M., et al.: TensorFlow: a system for large-scale machine learning. In: OSDI, vol. 16, pp. 265–283 (2016)

    Google Scholar 

  2. Anaconda: Anaconda documentation (2022). https://www.anaconda.com/products

  3. Asghari, M., Mirzapour Al-e-hashem, S.M.J.: Green vehicle routing problem: a state-of-the-art review. Int. J. Prod. Econ. 231, 107899 (2021). https://doi.org/10.1016/j.ijpe.2020.107899. https://www.sciencedirect.com/science/article/pii/S0925527320302607

  4. Barto, A.G., Sutton, R.S., Anderson, C.W.: Looking back on the actor-critic architecture. IEEE Trans. Syst. Man Cybern. Syst. 51(1), 40–50 (2021). https://doi.org/10.1109/TSMC.2020.3041775

    Article  Google Scholar 

  5. Borrero, I., Arias, M.: Deep Learning. Alonso Barba, Universidad de Huelva (2021). https://books.google.com.ar/books?id=kzsvEAAAQBAJ

  6. Clarke, G., Wright, J.W.: Scheduling of vehicles from a central depot to a number of delivery points. Oper. Res. 12(4), 568–581 (1964). http://www.jstor.org/stable/167703

  7. Ebrahimnejad, A., Verdegay, J.L.: Fuzzy Sets-Based Methods and Techniques for Modern Analytics. SFSC, vol. 364. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73903-8

    Book  MATH  Google Scholar 

  8. Flood, M.M.: The traveling-salesman problem. Oper. Res. 4(1), 61–75 (1956). http://www.jstor.org/stable/167517

  9. Garofalakis, M., Gehrke, J., Rastogi, R.: Data Stream Management: Processing High-Speed Data Streams. Data-Centric Systems and Applications. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-540-28608-0. https://books.google.com.ar/books?id=qiSpDAAAQBAJ

  10. Gorelick, M., Ozsvald, I.: High Performance Python: Practical Performant Programming for Humans. O’Reilly Media (2020). https://books.google.com.ar/books?id=kKjgDwAAQBAJ

  11. Hafner, D., Davidson, J., Vanhoucke, V.: TensorFlow agents: efficient batched reinforcement learning in TensorFlow. CoRR abs/1709.02878 (2017). http://arxiv.org/abs/1709.02878

  12. Haykin, S.: Neural Networks: A Comprehensive Foundation. Macmillan, New York (1994)

    Google Scholar 

  13. Huerta, I.I., Neira, D.A., Ortega, D.A., Varas, V., Godoy, J., Asín-Achá, R.: Improving the state-of-the-art in the traveling salesman problem: an anytime automatic algorithm selection. Expert Syst. Appl. 187, 115948 (2022). https://doi.org/10.1016/j.eswa.2021.115948. https://www.sciencedirect.com/science/article/pii/S0957417421013014

  14. Karp, R.M.: Reducibility among Combinatorial Problems, pp. 85–103. Springer, Boston (1972). https://doi.org/10.1007/978-1-4684-2001-2_9

  15. Kirk, D., Hwu, W.: Programming Massively Parallel Processors: A Hands-on Approach. Elsevier Science (2016)

    Google Scholar 

  16. NVIDIA: NVIDIA CUDA Compute Unified Device Architecture, Programming Guide. NVIDIA (2020)

    Google Scholar 

  17. NVIDIA: Nvidia: CUDA C++ Programming Guide, Design Guide. NVIDIA (2021)

    Google Scholar 

  18. Pacheco, P., Malensek, M.: An Introduction to Parallel Programming. Elsevier Science (2021). https://books.google.com.ar/books?id=rElkCwAAQBAJ

  19. Perumalla, K., Alam, M.: Design considerations for GPU-based mixed integer programming on parallel computing platforms, chap. 27. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3458744.3473366

  20. Pulido-López, D.G., García, M., Figueroa-García, J.C.: Fuzzy uncertainty in random variable generation: a cumulative membership function approach. In: Figueroa-García, J.C., López-Santana, E.R., Villa-Ramírez, J.L., Ferro-Escobar, R. (eds.) WEA 2017. CCIS, vol. 742, pp. 398–407. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66963-2_36

    Chapter  Google Scholar 

  21. Rashid, M.H., McAndrew, I.: An efficient GPU framework for parallelizing combinatorial optimization heuristics. In: 2020 International Conference on Advances in Computing and Communication Engineering (ICACCE), pp. 1–7 (2020). https://doi.org/10.1109/ICACCE49060.2020.9155072

  22. Russell, S.J., Norvig, P.: Inteligencia artificial: un enfoque moderno. Pearson Prentice Hall, Madrid (2004)

    Google Scholar 

  23. Rutkowski, L., Jaworski, M., Duda, P.: Stream Data Mining: Algorithms and Their Probabilistic Properties. SBD, vol. 56. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-13962-9 https://books.google.com.ar/books?id=P0-NDwAAQBAJ

    Book  MATH  Google Scholar 

  24. Schab, E.A., Casanova, C.A., Piccoli, M.F.: Reinforcement learning for VRP, April 2022. https://github.com/estebanschab/RL-VRP

  25. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). arXiv preprint arXiv:1707.06347

  26. Siddique, N., Adeli, H.: Computational Intelligence: Synergies of Fuzzy Logic, Neural Networks and Evolutionary Computing. Wiley (2013). https://books.google.com.ar/books?id=CbpbuA0jvVgC

  27. Singh, P., Manure, A.: Learn TensorFlow 2.0: Implement Machine Learning and Deep Learning Models with Python. Apress (2019). https://books.google.com.ar/books?id=3_rEDwAAQBAJ

  28. Soyata, T.: GPU Parallel Program Development Using CUDA. T. Francis, Abingdon (2018)

    Book  Google Scholar 

  29. Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press (2018)

    Google Scholar 

  30. Terzo, O., Martinovič, J.: HPC, Big Data, and AI Convergence Towards Exascale: Challenge and Vision. CRC Press (2022). https://books.google.com.ar/books?id=2NpXEAAAQBAJ

  31. Toomey, D.: Learning Jupyter 5: Explore Interactive Computing Using Python, Java, JavaScript, R, Julia, and JupyterLab, 2nd edn. Packt Publishing (2018). https://books.google.com.ar/books?id=8kZsDwAAQBAJ

  32. Varón-Gaviria, C.A., Barbosa-Fontecha, J.L., Figueroa-García, J.C.: Fuzzy uncertainty in random variable generation: an \(\alpha \)-cut approach. In: Huang, D.-S., Hussain, A., Han, K., Gromiha, M.M. (eds.) ICIC 2017. LNCS (LNAI), vol. 10363, pp. 264–273. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63315-2_23

    Chapter  Google Scholar 

  33. Wilt, N.: The CUDA Handbook: A Comprehensive Guide to GPU Programming. Addison Wesley (2020). https://books.google.com.ar/books?id=lUVQswEACAAJ

  34. Wintjen, M., Vlahutin, A.: Practical Data Analysis Using Jupyter Notebook: Learn How to Speak the Language of Data by Extracting Useful and Actionable Insights Using Python. Packt Publishing (2020). https://books.google.com.ar/books?id=tqTsDwAAQBAJ

  35. Zadeh, L.A.: Fuzzy logic, neural networks, and soft computing. Commun. ACM 37(3), 77–84 (1994). https://doi.org/10.1145/175247.175255

Download references

Author information

Authors and Affiliations

  1. Facultad Regional Concepción del Uruguay, Universidad Tecnológica Nacional, Buenos Aires, Argentina

    Esteban Schab & Carlos Casanova

  2. Universidad Nacional de San Luis, Ejército de los Andes 950, San Luis, Argentina

    Fabiana Piccoli

  3. Universidad Autónoma de Entre Ríos, 25 de Mayo 385, Concepción del Uruguay, Entre Ríos, Argentina

    Esteban Schab, Carlos Casanova & Fabiana Piccoli

Authors
  1. Esteban Schab
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Casanova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabiana Piccoli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Esteban Schab .

Editor information

Editors and Affiliations

  1. National University of La Plata, La Plata, Argentina

    Enzo Rucci

  2. National University of La Plata, La Plata, Argentina

    Marcelo Naiouf

  3. National University of La Plata, La Plata, Argentina

    Franco Chichizola

  4. National University of La Plata, La Plata, Argentina

    Laura De Giusti

  5. National University of La Plata, La Plata, Argentina

    Armando De Giusti

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schab, E., Casanova, C., Piccoli, F. (2022). Solving an Instance of a Routing Problem Through Reinforcement Learning and High Performance Computing. In: Rucci, E., Naiouf, M., Chichizola, F., De Giusti, L., De Giusti, A. (eds) Cloud Computing, Big Data & Emerging Topics. JCC-BD&ET 2022. Communications in Computer and Information Science, vol 1634. Springer, Cham. https://doi.org/10.1007/978-3-031-14599-5_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-14599-5_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-14598-8

  • Online ISBN: 978-3-031-14599-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics