Yujun Cui
ABSTRACT
Train automatic stop control (TASC) is one of the key techniques of Automatic train operation (ATO) to achieve high stopping precision. Aiming to improve accurate stopping performance, this paper proposes a novel TASC method based on double deep Q-network (DDQN) using knowledge from experienced driver to address time allocation of braking command. The knowledge is used for estimating a braking command to improve the learning efficiency, and DDQN determines the execution time of the command to avoid frequent switching of commands and ultimately reach better stopping decisions. The proposed method can achieve a probability of 100% and significantly outperforms 3 existing methods on the stopping errors within ± 0.30 m under high disturbances in the simulation platform, which is based on actual field data from the Beijing-Shenyang high-speed railway provided by cooperative enterprise.
- Jiateng Yin, Tao Tang, Lixing Yang, 2017. Research and development of automatic train operation for railway transportation systems: a survey. Transportation Research Part C: Emerging Technologies 85 (Sept. 2017), 548-572. https://doi.org/10.1016/j.trc.2017.09.009Google Scholar
- Kouki Yoshimoto, Kenji Kataoka, and Kiyotoshi Komaya. 2001. A feasibility study of train automatic stop control using range sensors. In Proceedings of the 2001 IEEE Intelligent Transportation Systems. IEEE, The Oakland, CA, 802-807. https://doi.org/10.1109/ITSC.2001.948763Google ScholarCross Ref
- Mohammad A. Sandidzadeh, Ali Khodadadi. 2011. Optimization of balise placement in a railway track using a vehicle, an odometer and genetic algorithm. Journal of Scientific and Industrial Research 70, 3 (Mar. 2011), 210-214. https://www.mendeley.com/catalogue/bcfa4e9c-5d74-31cb-a391-6047d94d5447/Google Scholar
- Jiateng Yin, Dewang Chen, Tao Tang, 2016. Balise arrangement optimization for train station parking via expert knowledge and genetic algorithm. Applied Mathematical Modelling 40, 19-20 (Oct. 2016), 8513-8529. https://doi.org/10.1016/j.apm.2016.04.015Google ScholarCross Ref
- Seiji Yasunobu, Shoji Miyamoto, and Hirokazu Ihara. 1983. A Fuzzy Control for Train Automatic Stop Control. Transactions of the Society of Instrument and Control Engineers 19, 11 (July. 1983), 873-880. http://dx.doi.org/10.9746/sicetr1965.19.873Google ScholarCross Ref
- Zhongsheng Hou, Yi Wang, Chenkun Yin, and Tao Tang. 2011. Terminal iterative learning control based station stop control of a train. International Journal of Control 84, 7 (July. 2011), 1263-1274. http://dx.doi.org/10.1080/00207179.2011.569030Google ScholarCross Ref
- Dewang Chen, Rong Chen, Yidong Li, and Tao Tang. 2013. Online Learning Algorithms for Train Automatic Stop Control Using Precise Location Data of Balises. IEEE Transactions on Intelligent Transportation Systems 14, 3 (Sept. 2013), 1526-1535. http://dx.doi.org/10.1109/TITS.2013.2265171Google ScholarDigital Library
- Peng Wu, Qingyuan Wang. 2014. Research of the automatic train stop control based on adaptive generalized predictive control. In Proceedings of the 33rd Chinese Control Conference. IEEE, Nanjing, China, 3399-3404. http://dx.doi.org/10.1109/ChiCC.2014.6895502Google ScholarCross Ref
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (Feb. 2015), 529–533. http://dx.doi.org/10.1038/nature14236Google ScholarCross Ref
- David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (Jan. 2016), 484-489. http://dx.doi.org/10.1038/nature16961Google ScholarCross Ref
- Rui Zhou, Shiji Song. 2018. Optimal Automatic Train Operation Via Deep Reinforcement Learning. In Proceedings of the 2018 Tenth International Conference on Advanced Computational Intelligence. IEEE, Xiamen, China, 103-108. http://dx.doi.org/10.1109/ICACI.2018.8377589Google ScholarCross Ref
- Miao Zhang, Qi Zhang, Yisheng Lv, Wenzhe Sun, Haiteng Wang. 2018. An AI based high-speed railway automatic train operation system analysis and design. In Proceedings of the 2018 International Conference on Intelligent Rail Transportation. IEEE, Singapore, Singapore, 1-5. http://dx.doi.org/10.1109/ICIRT.2018.8641650Google ScholarCross Ref
- Hado van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double q-learning. In Proceedings of the Thirtieth AAAI conference on artificial intelligence. AAAI, Phoenix, USA, 2094-2100. https://dl.acm.org/doi/abs/10.5555/3016100.3016191Google ScholarCross Ref
- Zhenyu Yu and Dewang Chen. 2011. Modeling and System Identification of the Braking System of Urban Rail Vehicles. Journal of the China Railway Society 33, 10 (Oct. 2011), 37-40. http://dx.doi.org/10.3969/j.issn.1001-8360.2011.10.007Google Scholar
- Hongge Guo and Keming Xie. 2014. Hammerstein Model and Parameters Identification of EMU Braking System. Journal of the China Railway Society 36, 4 (April. 2014), 48-53. http://dx.doi.org/10.3969/j.issn.1001-8360.2014.04.009Google Scholar
- Ruijun Cheng, Dewang Chen, Bao Cheng, Song Zheng. 2017. Intelligent driving methods based on expert knowledge and online optimization for high-speed trains. Expert Systems with Applications 87 (June. 2017), 228-239. http://dx.doi.org/10.1016/j.eswa.2017.06.006Google ScholarDigital Library
- Richard S. Sutton,Andrew G. Barto. 1998. Reinforcement learning: An introduction. John Wiley and Sons, Cambridge, MA.Google ScholarDigital Library
Recommendations
Deep Reinforcement Learning: From Q-Learning to Deep Q-Learning
Neural Information ProcessingAbstractAs the two hottest branches of machine learning, deep learning and reinforcement learning both play a vital role in the field of artificial intelligence. Combining deep learning with reinforcement learning, deep reinforcement learning is a method ...
Conversational Recommender System Using Deep Reinforcement Learning
RecSys '22: Proceedings of the 16th ACM Conference on Recommender SystemsDeep Reinforcement Learning (DRL) uses the best of both Reinforcement Learning and Deep Learning for solving problems which cannot be addressed by them individually. Deep Reinforcement Learning has been used widely for games, robotics etc. Limited work ...
Deep Reinforcement Learning based dynamic optimization of bus timetable
AbstractBus timetable optimization is a key issue to reduce operational cost of bus company and improve the transit service quality. Existing methods optimize the timetable offline. However, in practice, the short-term passenger flow may ...
Highlights- A reinforcement learning based method is proposed to optimize bus timetable.
- ...
Comments