Skip to main content
SpringerLink
Log in
Book cover

International Conference on Knowledge Science, Engineering and Management

KSEM 2021: Knowledge Science, Engineering and Management pp 101–113Cite as

Traffic Route Planning in Partially Observable Environment Using Actions Group Representation

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12816))

Abstract

We investigate the problem of optimal route planning formulized as Partially Observable Markov Decision Process (POMDP) [1]: Given a partially traffic-aware road network, we aim to find a route for agent vehicle such that the global travel time cost is minimized. In this paper, we show that the theory of group representation with its ability to make mechanism of \(\mathcal {A}\times \mathcal {S}\) (actions acting on states) computable efficiently, which is able to provide significant advantages in multi-step planning with information partially observable. Using the action group Representation, we build a more “visionary” system. Extensive experiments offer insight into the efficiency of proposed algorithms.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99–134 (1998)

    Article  MathSciNet  Google Scholar 

  2. Aklam, F., Osborn, W.: Dynamic group trip planning queries in spatial databases (2020)

    Google Scholar 

  3. Bazzan, A.L.C., Grunitzki, R.: A multiagent reinforcement learning approach to en-route trip building. In: 2016 International Joint Conference on Neural Networks (IJCNN) (2016)

    Google Scholar 

  4. Brechtel, S., Gindele, T., Dillmann, R.: Probabilistic decision-making under uncertainty for autonomous driving using continuous POMDPs. In: IEEE International Conference on Intelligent Transportation Systems (2014)

    Google Scholar 

  5. Cai, C.: Group representation theory for knowledge graph embedding (2019)

    Google Scholar 

  6. Cao, X., Chen, L., Cong, G., Xiao, X.: Keyword-aware optimal route search. Proc. VLDB Endowment 5, 1136–1147 (2012)

    Article  Google Scholar 

  7. Chen, M., Zhang, Y., Qiu, M., Guizani, N., Hao, Y.: SPHA: smart personal health advisor based on deep analytics. IEEE Commun. Mag. 56, 164–169 (2018)

    Article  Google Scholar 

  8. Chollet, F., et al.: Keras (2015). https://keras.io

  9. Collins, M.J.: Representations and Characters of Finite Groups. Cambridge University Press, Cambridge (1990)

    MATH  Google Scholar 

  10. Deisenroth, M.P., Rasmussen, C.E.: PILCO: a model-based and data-efficient approach to policy search (2011)

    Google Scholar 

  11. Dotoli, M., Hammadi, S., Jeribi, K.: A multi-agent decision support system for optimization of co-modal transportation route planning services. In: Decision and Control (2014)

    Google Scholar 

  12. Dummit, D.S., Foote, R.M.: Abstract Algebra (2004)

    Google Scholar 

  13. Fridman, L., Terwilliger, J., Jenik, B.: DeepTraffic: crowdsourced hyperparameter tuning of deep reinforcement learning systems for multi-agent dense traffic navigation. In: Neural Information Processing Systems (NIPS 2018) Deep Reinforcement Learning Workshop (2018)

    Google Scholar 

  14. Gai, K., Qiu, M.: Reinforcement learning-based content-centric services in mobile sensing. IEEE Netw. 32, 34–39 (2018)

    Article  Google Scholar 

  15. Gao, L., Du, Y., Li, H., Lin, G.: RotEqNet: rotation-equivariant network for fluid systems with symmetric high-order tensors (2020)

    Google Scholar 

  16. Graf, T., Platzner, M.: Adaptive playouts in Monte-Carlo tree search with policy-gradient reinforcement learning. In: Plaat, A., van den Herik, J., Kosters, W. (eds.) ACG 2015. LNCS, vol. 9525, pp. 1–11. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27992-3_1

    Chapter  Google Scholar 

  17. Gu, S., Holly, E., Lillicrap, T., Levine, S.: Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In: IEEE International Conference on Robotics and Automation (2017)

    Google Scholar 

  18. Gu, S., Lillicrap, T., Sutskever, I., Levine, S.: Continuous deep q-learning with model-based acceleration (2016)

    Google Scholar 

  19. Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)

    Google Scholar 

  20. Hu, Z., Zhu, M., Liu, P.: Adaptive cyber defense against multi-stage attacks using learning-based POMDP. ACM Trans. Privacy Secur. 24, 1–25 (2020)

    Google Scholar 

  21. Pierre, J.: Linear Representations of Finite Groups. Springer, Heidelberg (2008)

    Google Scholar 

  22. Kelly, S.: Basic Introduction to PyGame. Apress, New York (2016)

    Book  Google Scholar 

  23. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. Computer science (2014)

    Google Scholar 

  24. Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies. J. Mach. Learn. Res. 17, 1334–1373 (2015)

    MathSciNet  MATH  Google Scholar 

  25. Lu, H., Hu, H.: DensE: an enhanced non-abelian group representation for knowledge graph embedding (2020)

    Google Scholar 

  26. Luo, Y., Xu, H., Li, Y., Tian, Y., Darrell, T., Ma, T.: Algorithmic framework for model-based deep reinforcement learning with theoretical guarantees (2019)

    Google Scholar 

  27. Luong, N.C., Hoang, D.T., Gong, S., Niyato, D., Kim, D.I.: Applications of deep reinforcement learning in communications and networking: a survey. IEEE Commun. Surv. Tutor. 21, 3133–3174 (2019)

    Article  Google Scholar 

  28. Paden, B., Cap, M., Yong, S.Z., Yershov, D., Frazzoli, E.: A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Trans. Intell. Veh. 1, 33–55 (2016)

    Article  Google Scholar 

  29. Rane, K.P.: Design and development of IoT, web-server and ML-AVPR based intelligent humanoid robot for traffic assistance. Int. J. Adv. Trends Comput. Sci. Eng. 9, 1922–1929 (2020)

    Article  Google Scholar 

  30. Shang, S., Liu, J., Zheng, K., Lu, H., Pedersen, T.B., Wen, J.-R.: Planning unobstructed paths in traffic-aware spatial networks. GeoInformatica 19(4), 723–746 (2015). https://doi.org/10.1007/s10707-015-0227-9

    Article  Google Scholar 

  31. Sharifzadeh, M., Kolahdouzan, M., Shahabi, C.: The optimal sequenced route query. VLDB J. 17, 765–787 (2008)

    Article  Google Scholar 

  32. Sharifzadeh, S., Chiotellis, I., Triebel, R., Cremers, D.: Learning to drive using inverse reinforcement learning and deep q-networks (2016)

    Google Scholar 

  33. Shen, J., Zhao, H., Zhang, W., Yu, Y.: Model-based policy optimization with unsupervised model adaptation (2020)

    Google Scholar 

  34. Shridhar, M., Thomason, J., Gordon, D., Bisk, Y., Han, W.: ALFRED: a benchmark for interpreting grounded instructions for everyday tasks (2020)

    Google Scholar 

  35. Shutty, N., Wierzynski, C.: Learning irreducible representations of noncommutative lie groups (2020)

    Google Scholar 

  36. Alemzadeh, S., Moslemi, R., Sharma, R.: Adaptive traffic control with deep reinforcement learning: towards state-of-the-art and beyond (2020)

    Google Scholar 

  37. Soma, S.C., Hashem, T., Cheema, M.A., Samrose, S.: Trip planning queries with location privacy in spatial databases. World Wide Web 20(2), 205–236 (2016). https://doi.org/10.1007/s11280-016-0384-2

    Article  Google Scholar 

  38. Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992). https://doi.org/10.1007/BF00992698

    Article  MATH  Google Scholar 

  39. Weiler, M., Cesa, G.: General E(2)-equivariant steerable CNNs. arXiv (2019)

    Google Scholar 

  40. Xu, T., Wang, N., Lin, H., Sun, Z.: UAV autonomous reconnaissance route planning based on deep reinforcement learning. In: 2019 IEEE International Conference on Unmanned Systems (ICUS) (2019)

    Google Scholar 

  41. Xu, Y., et al.: Location-based top-k term querying over sliding window. In: Bouguettaya, A., et al. (eds.) WISE 2017. LNCS, vol. 10569, pp. 299–314. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68783-4_21

    Chapter  Google Scholar 

  42. Yu, H., Lian, X., Zhang, H., Xu, W.: Guided feature transformation (GFT): a neural language grounding module for embodied agents (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Guangzhou, China

    Minzhong Luo

  2. School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

    Minzhong Luo

  3. Brainnetome Center and National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Shan Yu

Authors
  1. Minzhong Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Minzhong Luo .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Han Qiu

  2. Ibaraki University, Hitachi, Japan

    Cheng Zhang

  3. University of Kentucky, Lexington, KY, USA

    Zongming Fei

  4. Texas A&M University – Commerce, Commerce, TX, USA

    Meikang Qiu

  5. Princeton University, Princeton, NJ, USA

    Sun-Yuan Kung

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Luo, M., Yu, S. (2021). Traffic Route Planning in Partially Observable Environment Using Actions Group Representation. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management . KSEM 2021. Lecture Notes in Computer Science(), vol 12816. Springer, Cham. https://doi.org/10.1007/978-3-030-82147-0_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-82147-0_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-82146-3

  • Online ISBN: 978-3-030-82147-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation