Skip to main content
SpringerLink
Log in

Making the Case for Safety of Machine Learning in Highly Automated Driving

  • Conference paper
  • First Online:
Computer Safety, Reliability, and Security (SAFECOMP 2017)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10489))

Included in the following conference series:

Abstract

This paper describes the challenges involved in arguing the safety of highly automated driving functions which make use of machine learning techniques. An assurance case structure is used to highlight the systems engineering and validation considerations when applying machine learning methods for highly automated driving. Particular focus is placed on addressing functional insufficiencies in the perception functions based on convolutional neural networks and possible types of evidence that can be used to mitigate against such risks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    See https://www.youtube.com/watch?v=u6aEYuemt0M for an introduction.

References

  1. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety. arXiv e-prints, June 2016

    Google Scholar 

  2. Attenberg, J., Ipeirotis, P., Provost, F.: Beat the machine: challenging humans to find a predictive model’s “unknown unknowns”. ACM J. Data Inf. Qual. 1(1), 1–17 (2014)

    Google Scholar 

  3. Bergenhem, C., Johansson, R., Söderberg, A., Nilsson, J., Tryggvesson, J., Törngren, M., Ursing, S.: How to reach complete safety requirement refinement for autonomous vehicles. Technical report, CARS 2015 - Critical Automotive applications: Robustness & Safety, September 2015, Paris, France (2015)

    Google Scholar 

  4. Binder, A., Bach, S., Montavon, G., Müller, K.R., Samek, W.: Layer-wise relevance propagation for deep neural network architectures. In: Kim, K., Joukov, N. (eds.) Information Science and Applications (ICISA). LNEE, vol. 376, pp. 913–922. Springer, Singapore (2016). doi:10.1007/978-981-10-0557-2_87

    Google Scholar 

  5. Castelvecchi, D.: Can we open the black box of AI? Nature 538(7623), 20–23 (2016). http://www.nature.com/news/can-we-open-the-black-box-of-ai-1.20731

  6. Chen, C., Seff, A., Kornhauser, A., Xiao, J.: Deepdriving: learning affordance for direct perception in autonomous driving. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), pp. 2722–2730. IEEE (2015)

    Google Scholar 

  7. Christmann, A., Steinwart, I.: Support Vector Machines. Springer, Heidelberg (2008)

    MATH  Google Scholar 

  8. Fawzi, A., Fawzi, O., Frossard, P.: Analysis of classifiers’ robustness to adversarial perturbations. arXiv:1502.02590 (2015)

  9. Feinberg, E.A., Shwartz, A. (eds.): International Series in Operations Research & Management Science, vol. 40. Springer, Heidelberg (2002)

    Google Scholar 

  10. Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2), 131–163 (1997)

    Article  MATH  Google Scholar 

  11. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  12. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv:1412.6572 (2015)

  13. Heckemann, K., Gesell, M., Pfister, T., Berns, K., Schneider, K., Trapp, M.: Safe automotive software. In: König, A., Dengel, A., Hinkelmann, K., Kise, K., Howlett, R.J., Jain, L.C. (eds.) KES 2011. LNCS, vol. 6884, pp. 167–176. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23866-6_18

    Chapter  Google Scholar 

  14. Hendricks, L.A., Akata, Z., Rohrbach, M., Donahue, J., Schiele, B., Darrell, T.: Generating visual explanations. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 3–19. Springer, Cham (2016). doi:10.1007/978-3-319-46493-0_1. http://arxiv.org/abs/1603.08507

    Chapter  Google Scholar 

  15. IEEE: IEEE standard adoption of ISO/IEC 15026–1 - systems and software engineering - systems and software assurance (2014)

    Google Scholar 

  16. ISO: ISO 26262: Road vehicles - functional safety (2011)

    Google Scholar 

  17. Kaufman, S., Rosset, S., Perlich, C.: Leakage in data mining: Formulation, detection, and avoidance. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 556–563. ACM (2011)

    Google Scholar 

  18. Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? CoRR abs/1703.04977 (2017). http://arxiv.org/abs/1703.04977

  19. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  20. Mohan, N., Törngren, M., Izosimov, V., Kaznov, V., Roos, P., Svahn, J., Gustavsson, J., Nesic, D.: Challenges in architecting fully automated driving; with an emphasis on heavy commercial vehicles. In: 2016 Workshop on Automotive Systems/Software Architectures (2016)

    Google Scholar 

  21. Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, pp. 427–436 (2015)

    Google Scholar 

  22. Piziali, A.: Functional Verification Coverage Measurement and Analysis. Springer, Heidelberg (2008)

    Google Scholar 

  23. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  24. Richter, S.R., Vineet, V., Roth, S., Koltun, V.: Playing for data: ground truth from computer games. arXiv:1608.02192 (2016)

  25. SAE: J3016, taxonomy and definitions for terms related to on-road motor vehicle automated driving systems (2013)

    Google Scholar 

  26. Tas, Ö.S., Kuhnt, F., Zöllner, J.M., Stiller, C.: Functional system architectures towards fully automated driving. In: 2016 IEEE Intelligent Vehicles Symposium. IEEE (2016)

    Google Scholar 

  27. Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.F. Dennison, D.: Hidden technical debt in machine learning systems. Advances in Neural Information Processing Systems, 28 (NIPS 2015) (2015)

    Google Scholar 

  28. Shalev-Shwartz, S., Shashua, A.: On the sample complexity of end-to-end training vs. semantic abstraction training. In: arXiv:1604.06915 (2016)

  29. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: arXiv:1312.6034 (2014)

  30. Kelly, T., Weaver, R.: The goal structuring notation - a safety argument notation. In: Proceedings of the DSN 2004 Workshop on Assurance Cases (2004)

    Google Scholar 

  31. Varshney, K.: Engineering safety in machine learning. ArXiv e-prints, January 2016

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Corporate Research, Robert Bosch GmbH, Renningen, Germany

    Simon Burton, Lydia Gauerhof & Christian Heinzemann

Authors
  1. Simon Burton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lydia Gauerhof
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Heinzemann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Burton .

Editor information

Editors and Affiliations

  1. Fondazione Bruno Kessler, Trento, Italy

    Stefano Tonetta

  2. Austrian Institute of Technology GmbH, Vienna, Austria

    Erwin Schoitsch

  3. Thales Transportation Systems GmbH, Ditzingen, Germany

    Friedemann Bitsch

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Burton, S., Gauerhof, L., Heinzemann, C. (2017). Making the Case for Safety of Machine Learning in Highly Automated Driving. In: Tonetta, S., Schoitsch, E., Bitsch, F. (eds) Computer Safety, Reliability, and Security . SAFECOMP 2017. Lecture Notes in Computer Science(), vol 10489. Springer, Cham. https://doi.org/10.1007/978-3-319-66284-8_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66284-8_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66283-1

  • Online ISBN: 978-3-319-66284-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation