Abstract
Adaptive cruise control (ACC) seeks intelligent and adaptive methods for longitudinal control of the cars. Since more than a decade, high-end cars have been equipped with ACC typically through carefully designed model-based controllers. Unlike the traditional ACC, we propose a reinforcement learning based approach – RL-ACC. We present the RL-ACC and its experimental results from the automotive-grade car simulators. Thus, we obtain a controller which requires minimal domain knowledge, is intuitive in its design, can accommodate uncertainties, can mimic human-like behaviour and may enable human-trust in the automated system. All these aspects are crucial for a fully autonomous car and we believe reinforcement learning based ACC is a step towards that direction.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The vehicle is equipped with both long and short range radars with limits [80, 240] and [0.2 100] m respectively. Unlike the problems like parking, the ACC does not require very close range detection. Here, sensor fusion is used to homogenise the readings from both the radars.
References
Drivecore - ces2018. http://wardsauto.com/technology/visteon-looks-play-big-role-autonomous-vehicles-drivecore/. Accessed Feb 2018
Oktal - simulation in motion. http://www.oktal.fr/en/automotive/range-of-simulators/software. Accessed Feb 2018
Vtd - virtual test drive. https://vires.com/vtd-vires-virtual-test-drive/. Accessed Feb 2018
Bando, M., Hasebe, K., Nakayama, A., Shibata, A., Sugiyama, Y.: Dynamical model of traffic congestion and numerical simulation. Phys. Rev. E 51(2), 1035 (1995)
Boer, E.R.: Car following from the drivers perspective. Transp. Res. Part F: Traffic Psychol. Behav. 2(4), 201–206 (1999)
Brackstone, M., McDonald, M.: Car-following: a historical review. Transp. Res. Part F: Traffic Psychol. Behav. 2(4), 181–196 (1999)
Gazis, D.C., Herman, R., Rothery, R.W.: Nonlinear follow-the-leader models of traffic flow. Oper. Res. 9(4), 545–567 (1961)
Gipps, P.G.: A behavioural car-following model for computer simulation. Transp. Res. Part B: Methodol. 15(2), 105–111 (1981)
Gray, R., Regan, D.: Accuracy of estimating time to collision using binocular and monocular information. Vision Res. 38(4), 499–512 (1998)
Helly, W.: Simulation of bottlenecks in single lane traffic flow, presentation at the symposium on theory of traffic flow. Research laboratories, General Motors, New York, pp. 207–238 (1959)
Lagoudakis, M.G., Parr, R.: Least-squares policy iteration. J. Mach. Learn. Res. 4(Dec), 1107–1149 (2003)
Littman, M.L., Dean, T.L., Leslie, P.K.: On the complexity of solving markov decision problems, pp. 394–402. Morgan Kaufmann Publishers Inc. (1995)
Michaels, R.M.: Perceptual factors in car following. In: Proceedings of the 2nd International Symposium on the Theory of Road Traffic Flow, London, England, OECD (1963)
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv preprintarXiv:1312.5602 (2013)
Moody, J., Saffell, M.: Learning to trade via direct reinforcement. IEEE Trans. Neural Netw. 12(4), 875–889 (2001)
Nagel, K., Schreckenberg, M.: A cellular automaton model for freeway traffic. Journal de physique I 2(12), 2221–2229 (1992)
Newell, G.F.: A simplified car-following theory: a lower order model. Transp. Res. Part B: Methodol. 36(3), 195–205 (2002)
Peters, J., Vijayakumar, S., Schaal, S.: Reinforcement learning for humanoid robotics. In: Proceedings of the Third IEEE-RAS International Conference on Humanoid Robots, pp. 1–20 (2003)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT Press Cambridge, Cambridge (1998)
Treiber, M., Hennecke, A., Helbing, D.: Congested traffic states in empirical observations and microscopic simulations. Phys. Rev. E 62(2), 1805 (2000)
Wiedemann, R.: Simulation des straßenverkehrsflusses. schriftenreihe heft 8. Institute for Transportation Science, University of Karlsruhe, Germany (1994)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Pathak, S., Bag, S., Nadkarni, V. (2019). A Generalised Method for Adaptive Longitudinal Control Using Reinforcement Learning. In: Strand, M., Dillmann, R., Menegatti, E., Ghidoni, S. (eds) Intelligent Autonomous Systems 15. IAS 2018. Advances in Intelligent Systems and Computing, vol 867. Springer, Cham. https://doi.org/10.1007/978-3-030-01370-7_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-01370-7_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01369-1
Online ISBN: 978-3-030-01370-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)