skip to main content
10.1145/3426826.3426833acmotherconferencesArticle/Chapter ViewAbstractPublication PagesmlmiConference Proceedingsconference-collections
research-article

Knowledge-based Deep Reinforcement Learning for Train Automatic Stop Control of High-Speed Railway

Published: 17 December 2020 Publication History

Abstract

Train automatic stop control (TASC) is one of the key techniques of Automatic train operation (ATO) to achieve high stopping precision. Aiming to improve accurate stopping performance, this paper proposes a novel TASC method based on double deep Q-network (DDQN) using knowledge from experienced driver to address time allocation of braking command. The knowledge is used for estimating a braking command to improve the learning efficiency, and DDQN determines the execution time of the command to avoid frequent switching of commands and ultimately reach better stopping decisions. The proposed method can achieve a probability of 100% and significantly outperforms 3 existing methods on the stopping errors within ± 0.30 m under high disturbances in the simulation platform, which is based on actual field data from the Beijing-Shenyang high-speed railway provided by cooperative enterprise.

References

[1]
Jiateng Yin, Tao Tang, Lixing Yang, 2017. Research and development of automatic train operation for railway transportation systems: a survey. Transportation Research Part C: Emerging Technologies 85 (Sept. 2017), 548-572. https://doi.org/10.1016/j.trc.2017.09.009
[2]
Kouki Yoshimoto, Kenji Kataoka, and Kiyotoshi Komaya. 2001. A feasibility study of train automatic stop control using range sensors. In Proceedings of the 2001 IEEE Intelligent Transportation Systems. IEEE, The Oakland, CA, 802-807. https://doi.org/10.1109/ITSC.2001.948763
[3]
Mohammad A. Sandidzadeh, Ali Khodadadi. 2011. Optimization of balise placement in a railway track using a vehicle, an odometer and genetic algorithm. Journal of Scientific and Industrial Research 70, 3 (Mar. 2011), 210-214. https://www.mendeley.com/catalogue/bcfa4e9c-5d74-31cb-a391-6047d94d5447/
[4]
Jiateng Yin, Dewang Chen, Tao Tang, 2016. Balise arrangement optimization for train station parking via expert knowledge and genetic algorithm. Applied Mathematical Modelling 40, 19-20 (Oct. 2016), 8513-8529. https://doi.org/10.1016/j.apm.2016.04.015
[5]
Seiji Yasunobu, Shoji Miyamoto, and Hirokazu Ihara. 1983. A Fuzzy Control for Train Automatic Stop Control. Transactions of the Society of Instrument and Control Engineers 19, 11 (July. 1983), 873-880. http://dx.doi.org/10.9746/sicetr1965.19.873
[6]
Zhongsheng Hou, Yi Wang, Chenkun Yin, and Tao Tang. 2011. Terminal iterative learning control based station stop control of a train. International Journal of Control 84, 7 (July. 2011), 1263-1274. http://dx.doi.org/10.1080/00207179.2011.569030
[7]
Dewang Chen, Rong Chen, Yidong Li, and Tao Tang. 2013. Online Learning Algorithms for Train Automatic Stop Control Using Precise Location Data of Balises. IEEE Transactions on Intelligent Transportation Systems 14, 3 (Sept. 2013), 1526-1535. http://dx.doi.org/10.1109/TITS.2013.2265171
[8]
Peng Wu, Qingyuan Wang. 2014. Research of the automatic train stop control based on adaptive generalized predictive control. In Proceedings of the 33rd Chinese Control Conference. IEEE, Nanjing, China, 3399-3404. http://dx.doi.org/10.1109/ChiCC.2014.6895502
[9]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (Feb. 2015), 529–533. http://dx.doi.org/10.1038/nature14236
[10]
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (Jan. 2016), 484-489. http://dx.doi.org/10.1038/nature16961
[11]
Rui Zhou, Shiji Song. 2018. Optimal Automatic Train Operation Via Deep Reinforcement Learning. In Proceedings of the 2018 Tenth International Conference on Advanced Computational Intelligence. IEEE, Xiamen, China, 103-108. http://dx.doi.org/10.1109/ICACI.2018.8377589
[12]
Miao Zhang, Qi Zhang, Yisheng Lv, Wenzhe Sun, Haiteng Wang. 2018. An AI based high-speed railway automatic train operation system analysis and design. In Proceedings of the 2018 International Conference on Intelligent Rail Transportation. IEEE, Singapore, Singapore, 1-5. http://dx.doi.org/10.1109/ICIRT.2018.8641650
[13]
Hado van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double q-learning. In Proceedings of the Thirtieth AAAI conference on artificial intelligence. AAAI, Phoenix, USA, 2094-2100. https://dl.acm.org/doi/abs/10.5555/3016100.3016191
[14]
Zhenyu Yu and Dewang Chen. 2011. Modeling and System Identification of the Braking System of Urban Rail Vehicles. Journal of the China Railway Society 33, 10 (Oct. 2011), 37-40. http://dx.doi.org/10.3969/j.issn.1001-8360.2011.10.007
[15]
Hongge Guo and Keming Xie. 2014. Hammerstein Model and Parameters Identification of EMU Braking System. Journal of the China Railway Society 36, 4 (April. 2014), 48-53. http://dx.doi.org/10.3969/j.issn.1001-8360.2014.04.009
[16]
Ruijun Cheng, Dewang Chen, Bao Cheng, Song Zheng. 2017. Intelligent driving methods based on expert knowledge and online optimization for high-speed trains. Expert Systems with Applications 87 (June. 2017), 228-239. http://dx.doi.org/10.1016/j.eswa.2017.06.006
[17]
Richard S. Sutton,Andrew G. Barto. 1998. Reinforcement learning: An introduction. John Wiley and Sons, Cambridge, MA.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
MLMI '20: Proceedings of the 2020 3rd International Conference on Machine Learning and Machine Intelligence
September 2020
138 pages
ISBN:9781450388344
DOI:10.1145/3426826
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 December 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Deep reinforcement learning
  2. Double deep Q-network
  3. Knowledge
  4. Train automatic stop control

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • National Key Research and Development Program of China under Grant
  • BNRist Program under Grants
  • National Natural Science Foundation of China under Grant

Conference

MLMI '20

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 82
    Total Downloads
  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media