Skip to main content
SpringerLink
Log in

End-to-End Learning of Autonomous Vehicle Lateral Control via MPC Training

  • Conference paper
  • First Online:
Intelligent Autonomous Systems 16 (IAS 2021)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 412))

Included in the following conference series:

Abstract

One of the main requirements of an autonomous vehicle is the ability to maintain its trajectory within the road lane. This task is generally performed utilizing vision data, processed using convolutional neural networks or classical computer vision algorithms to extract a road mask. A software pipeline then analyzes this mask to retrieve the vehicle’s relative state. This process is composed of many components that need to be tuned to achieve good results. What is proposed in this paper is instead an end-to-end solution able to infer the steering command directly from camera images. Differently from the classical end-to-end machine-learning approaches, the architecture is not trained using as ground truth the car data from a human driver, but instead the output of a control algorithm. The network then does not mimic a specific human behavior but learns how to achieve the optimal trajectory computed by the algorithm in an end-to-end fashion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Dataset available at: http://airlab.deib.polimi.it/datasets-and-tools/.

References

  1. SAE, et al.: Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems. SAE Stan. J. 3016, 1–16 (2014)

    Google Scholar 

  2. Cudrano, P., Mentasti, S., Matteucci, M., Bersani, M., Arrigoni, S., Cheli, F.: Advances in centerline estimation for autonomous lateral control. In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1415–1422 (2020). https://doi.org/10.1109/IV47402.2020.9304729

  3. Bar Hillel, A., Lerner, R., Levi, D., Raz, G.: Recent progress in road and lane detection: a survey. Mach. Vis. Appl. 25(3), 727–745 (2012). https://doi.org/10.1007/s00138-011-0404-2

    Article  Google Scholar 

  4. Zou, Q., Jiang, H., Dai, Q., Yue, Y., Chen, L., Wang, Q.: Robust lane detection from continuous driving scenes using deep neural networks. IEEE Trans. Veh. Technol. 69(1), 41–54 (2019)

    Article  Google Scholar 

  5. Bojarski, M., et al.: End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016)

  6. Pomerleau, D.A.: Alvinn: An autonomous land vehicle in a neural network. In: Advances in Neural Information Processing Systems, pp. 305–313 (1989)

    Google Scholar 

  7. Bojarski, M., et al.: Explaining how a deep neural network trained with end-to-end learning steers a car. arXiv preprint arXiv:1704.07911 (2017)

  8. Jhung, J., Bae, I., Moon, J., Kim, T., Kim, J., Kim, S.: End-to-end steering controller with CNN-based closed-loop feedback for autonomous vehicles. In: 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 617–622. IEEE (2018)

    Google Scholar 

  9. Huang, Z., Zhang, J., Tian, R., Zhang, Y.: End-to-end autonomous driving decision based on deep reinforcement learning. In: 2019 5th International Conference on Control, Automation and Robotics (ICCAR), pp 658–662. IEEE (2019)

    Google Scholar 

  10. Hou, Y., Hornauer, S., Zipser, K.: Fast recurrent fully convolutional networks for direct perception in autonomous driving. arXiv preprint arXiv:1711.06459 (2017)

  11. Eraqi, H.M., Moustafa, M.N., Honer, J.: End-to-end deep learning for steering autonomous vehicles considering temporal dependencies. arXiv preprint arXiv:1710.03804 (2017)

  12. Haselberger, J., Chen, J., Schick, B.: Deep learning for lateral vehicle control – an end-to-end trained multi-fusion steering model. In: 10th International Munich Chassis Symposium 2019. P, pp. 201–227. Springer, Wiesbaden (2020). https://doi.org/10.1007/978-3-658-26435-2_17

    Chapter  Google Scholar 

  13. Onishi, T., Motoyoshi, T., Suga, Y., Mori, H., Ogata, T.: End-to-end learning method for self-driving cars with trajectory recovery using a path-following function. In: 2019 International Joint Conference on Neural Networks, IEEE (2019)

    Google Scholar 

  14. Yang, Z., Zhang, Y., Yu, J., Cai, J., Luo, J.: End-to-end multi-modal multi-task vehicle control for self-driving cars with visual perceptions. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 2289–2294. IEEE (2018)

    Google Scholar 

  15. Zhang, M., Zhang, Y., Zhang, L., Liu, C., Khurshid, S.: Deeproad: gan-based metamorphic testing and input validation framework for autonomous driving systems. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. ASE 2018, New York, NY, USA, pp. 132-142 (2018)

    Google Scholar 

  16. Codevilla, F., Miiller, M., López, A., Koltun, V., Dosovitskiy, A.: End-to-end driving via conditional imitation learning. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–9. IEEE (2018)

    Google Scholar 

  17. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: Carla: an open urban driving simulator. arXiv preprint arXiv:1711.03938 (2017)

  18. Hubschneider, C., Bauer, A., Weber, M., Zöllner, J.M.: Adding navigation to the equation: turning decisions for end-to-end vehicle control. In: 2017 IEEE 20th International Conference on Intelligent Transportation Systems, IEEE (2017)

    Google Scholar 

  19. Pan, Y., et al.: Agile autonomous driving using end-to-end deep imitation learning. arXiv preprint arXiv:1709.07174 (2017)

  20. Viswanath, P., Nagori, S., Mody, M., Mathew, M., Swami, P.: End to end learning based self-driving using jacintonet. In: 2018 IEEE 8th International Conference on Consumer Electronics-Berlin (ICCE-Berlin), pp. 1–4. IEEE (2018)

    Google Scholar 

  21. Tian, Y., Pei, K., Jana, S., Ray, B.: Deeptest: Automated testing of deep-neural-network-driven autonomous cars. In: Proceedings of the 40th International Conference on Software Engineering, pp. 303–314 (2018)

    Google Scholar 

  22. Liniger, A., Domahidi, A., Morari, M.: Optimization-based autonomous racing of 1:43 scale RC cars. Optimal Control Appl. Methods 36(5), 628–647 (2014)

    Google Scholar 

  23. Frasch, J.V., et al.: An auto-generated nonlinear MPC algorithm for real-time obstacle avoidance of ground vehicles. In: 2013 European Control Conference (ECC), IEEE (2013)

    Google Scholar 

  24. Falcone, P., Borrelli, F., Asgari, J., Tseng, H.E., Hrovat, D.: Predictive active steering control for autonomous vehicle systems. IEEE Trans. Control Syst. Technol. 15(3), 566–580 (2007)

    Article  Google Scholar 

  25. Gao, Y., Lin, T., Borrelli, F., Tseng, E., Hrovat, D.: Predictive control of autonomous ground vehicles with obstacle avoidance on slippery roads. Dyn. Syst. Control Conf. 44175, 265–272 (2010)

    Google Scholar 

  26. Falcone, P., Borrelli, F., Asgari, J., Tseng, H.E., Hrovat, D.: Low complexity MPC schemes for integrated vehicle dynamics control problems. In: 9th International Symposium on Advanced Vehicle Control (2008)

    Google Scholar 

  27. Diehl, M., Gros, S.: Numerical optimal control. OPTEC (2011)

    Google Scholar 

  28. Houska, B., Ferreau, H.J., Diehl, M.: ACADO toolkit-an open-source framework for automatic control and dynamic optimization. Optimal Control Appl. Methods 32(3), 298–312 (2011)

    Article  MathSciNet  Google Scholar 

  29. Ferreau, H., Kirches, C., Potschka, A., Bock, H., Diehl, M.: qpOASES: a parametric active-set algorithm for quadratic programming. Math. Program. Comput. 6(4), 327–363 (2014)

    Article  MathSciNet  Google Scholar 

  30. Bersani, M., Vignati, M., Mentasti, S., Arrigoni, S., Cheli, F.: Vehicle state estimation based on Kalman filters. In: 2019 AEIT International Conference of Electrical and Electronic Technologies for Automotive (AEIT AUTOMOTIVE), July 2019

    Google Scholar 

  31. Vignati, M., Tarsitano, D., Cheli, F.: On how to transform a commercial electric quadricycle into a full autonomously actuated vehicle. In: 14th International Symposium on Advanced Vehicle Control (AVEC 2018), pp. 1–7 (2018)

    Google Scholar 

  32. Vignati, M., Tarsitano, D., Bersani, M., Cheli, F.: Autonomous steer actuation for an urban quadricycle. In: 2018 International Conference of Electrical and Electronic Technologies for Automotive, pp. 1–5, July 2018

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics Information and Bioengineering of Politecnico di Milano, P.zza Leonardo da Vinci 32, Milan, Italy

    Simone Mentasti & Matteo Matteucci

  2. Department of Mechanical Engineering of Politecnico di Milano, via La Masa 1, Milan, Italy

    Mattia Bersani, Stefano Arrigoni & Federico Cheli

Authors
  1. Simone Mentasti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mattia Bersani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefano Arrigoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matteo Matteucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Federico Cheli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simone Mentasti .

Editor information

Editors and Affiliations

  1. Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore

    Marcelo H. Ang Jr

  2. Department of Precision Engineering, University of Tokyo, Tokyo, Japan

    Hajime Asama

  3. Singapore Institute of Manufacturing Technology, Singapore, Singapore

    Wei Lin

  4. Engineering Product Development, Singapore University of Technology and Design, Singapore, Singapore

    Shaohui Foong

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mentasti, S., Bersani, M., Arrigoni, S., Matteucci, M., Cheli, F. (2022). End-to-End Learning of Autonomous Vehicle Lateral Control via MPC Training. In: Ang Jr, M.H., Asama, H., Lin, W., Foong, S. (eds) Intelligent Autonomous Systems 16. IAS 2021. Lecture Notes in Networks and Systems, vol 412. Springer, Cham. https://doi.org/10.1007/978-3-030-95892-3_15

Download citation

Publish with us

Policies and ethics

Search

Navigation