Skip to main content
SpringerLink
Log in

Improving ML Safety with Partial Specifications

  • Conference paper
  • First Online:
Computer Safety, Reliability, and Security (SAFECOMP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11699))

Included in the following conference series:

Abstract

Advanced autonomy features of vehicles are typically difficult or impossible to specify precisely and this has led to the rise of machine learning (ML) from examples as an alternative implementation approach to traditional programming. Developing software without specifications sacrifices the ability to effectively verify the software yet this is a key component of safety assurance. In this paper, we suggest that while complete specifications may not be possible, partial specifications typically are and these could be used with ML to strengthen safety assurance. We review the types of partial specifications that are applicable for these problems and discuss the places in the ML development workflow that they could be used to improve the safety of ML-based components.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bhattacharyya, S., Cofer, D., Musliner, D., Mueller, J., Engstrom, E.: Certification considerations for adaptive systems. In: 2015 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 270–279. IEEE (2015)

    Google Scholar 

  2. Cohen, T., Welling, M.: Group equivariant convolutional networks. In: International Conference on Machine Learning, pp. 2990–2999 (2016)

    Google Scholar 

  3. Cooke, D., Gates, A., Demirörs, E., Demirörs, O., Tanik, M.M., Krämer, B.: Languages for the specification of software. J. Syst. Softw. 32(3), 269–308 (1996)

    Article  Google Scholar 

  4. Czarnecki, K., Salay, R.: Towards a framework to manage perceptual uncertainty for safe automated driving. In: Gallina, B., Skavhaug, A., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2018. LNCS, vol. 11094, pp. 439–445. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99229-7_37

    Chapter  Google Scholar 

  5. Dreossi, T., Donzé, A., Seshia, S.A.: Compositional falsification of cyber-physical systems with machine learning components. In: Barrett, C., Davies, M., Kahsai, T. (eds.) NFM 2017. LNCS, vol. 10227, pp. 357–372. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57288-8_26

    Chapter  Google Scholar 

  6. Dwarakanath, A., et al.: Identifying implementation bugs in machine learning based image classifiers using metamorphic testing. In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 118–128. ACM (2018)

    Google Scholar 

  7. Gehr, T., Mirman, M., Drachsler-Cohen, D., Tsankov, P., Chaudhuri, S., Vechev, M.: Ai2: safety and robustness certification of neural networks with abstract interpretation. In: 2018 IEEE Symposium on Security and Privacy (SP), pp. 3–18. IEEE (2018)

    Google Scholar 

  8. Harnad, S.: The symbol grounding problem. Physica D 42(1–3), 335–346 (1990)

    Article  Google Scholar 

  9. Hinton, G.E., Sabour, S., Frosst, N.: Matrix capsules with EM routing. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada. Conference Track Proceedings, 30 April–3 May 2018. https://openreview.net/forum?id=HJWLfGWRb

  10. International Organization for Standardization: ISO 26262: Road Vehicles - Functional Safety, 2nd edition (2018)

    Google Scholar 

  11. International Organization for Standardization: ISO/AWI PAS 21448: Road Vehicles - Safety of the Intended Functionality, 1st Edition (2019)

    Google Scholar 

  12. Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 97–117. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_5

    Chapter  Google Scholar 

  13. Koopman, P., Wagner, M.: Challenges in autonomous vehicle testing and validation. SAE Int. J. Transp. Saf. 4(1), 15–24 (2016)

    Article  Google Scholar 

  14. Ku, J., Mozifian, M., Lee, J., Harakeh, A., Waslander, S.L.: Joint 3D proposal generation and object detection from view aggregation. In: 2018 IEEE/RSJ IROS, pp. 1–8. IEEE (2018)

    Google Scholar 

  15. Lakoff, G.: Women, Fire, and Dangerous Things: What Categories Reveal About the Mind. University of Chicago press, Chicago (1987)

    Google Scholar 

  16. Lamsweerde, A.V.: Formal specification: a roadmap. In: Proceedings of the Conference on the Future of Software Engineering, pp. 147–159. ACM (2000)

    Google Scholar 

  17. Liu, M.Y., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. In: Advances in Neural Information Processing Systems, pp. 700–708 (2017)

    Google Scholar 

  18. Meel, K.S., et al.: Constrained sampling and counting: universal hashing meets SAT solving. In: Workshops at the Thirtieth AAAI Conference on Artificial Intelligence (2016)

    Google Scholar 

  19. Meyer, B.: Applying ‘design by contract’. Computer 25(10), 40–51 (1992)

    Article  Google Scholar 

  20. Reiter, R.: A logic for default reasoning. Artif. Intell. 13(1–2), 81–132 (1980)

    Article  MathSciNet  Google Scholar 

  21. Rouder, J.N., Ratcliff, R.: Comparing exemplar and rule-based theories of categorization. Curr. Dir. Psychol. Sci. 15(1), 9–13 (2006)

    Article  Google Scholar 

  22. von Rueden, L., Mayer, S., Garcke, J., Bauckhage, C., Schuecker, J.: Informed machine learning-towards a taxonomy of explicit integration of knowledge into machine learning. arXiv preprint arXiv:1903.12394 (2019)

  23. Salay, R., Czarnecki, K.: Using machine learning safely in automotive software: An assessment and adaption of software process requirements in ISO 26262. arXiv preprint arXiv:1808.01614 (2018)

  24. Salay, R., Queiroz, R., Czarnecki, K.: An Analysis of ISO 26262: Machine Learning and Safety in Automotive Software. SAE Technical Paper (2018)

    Google Scholar 

  25. Seshia, S.A., Sadigh, D., Sastry, S.S.: Towards verified artificial intelligence. arXiv preprint arXiv:1606.08514 (2016)

  26. Sha, L.: Using simplicity to control complexity. IEEE Softw. 4, 20–28 (2001)

    Google Scholar 

  27. Spanfelner, B., Richter, D., Ebel, S., Wilhelm, U., Branz, W., Patz, C.: Challenges in applying the ISO 26262 for driver assistance systems. Tagung Fahrerassistenz, München 15(16), 2012 (2012)

    Google Scholar 

  28. Varshney, K.R.: Engineering safety in machine learning. arXiv preprint arXiv:1601.04126 (2016)

  29. Vedaldi, A., Blaschko, M., Zisserman, A.: Learning equivariant structured output SVM regressors. In: Proceedings of 2011 International Conference on Computer Vision, pp. 959–966. IEEE (2011)

    Google Scholar 

  30. Wang, J., Perez, L.: The effectiveness of data augmentation in image classification using deep learning. In: Convolutional Neural Networks Vision Recognition (2017)

    Google Scholar 

  31. Wong, S.C., Gatt, A., Stamatescu, V., McDonnell, M.D.: Understanding data augmentation for classification: when to warp? In: 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–6. IEEE (2016)

    Google Scholar 

  32. Worrall, D.E., Garbin, S.J., Turmukhambetov, D., Brostow, G.J.: Harmonic networks: deep translation and rotation equivariance. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5028–5037 (2017)

    Google Scholar 

  33. Xu, J., Zhang, Z., Friedman, T., Liang, Y., Broeck, G.V.D.: A semantic loss function for deep learning with symbolic knowledge. arXiv preprint arXiv:1711.11157 (2017)

  34. Yan, J., Zhang, X., Lei, Z., Liao, S., Li, S.Z.: Robust multi-resolution pedestrian detection in traffic scenes. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3033–3040. IEEE (2013)

    Google Scholar 

Download references

Acknowledgements

We would like to thank Mark Costin for insightful comments that have contributed to this work.

Author information

Authors and Affiliations

  1. University of Waterloo, Waterloo, Canada

    Rick Salay & Krzysztof Czarnecki

Authors
  1. Rick Salay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Krzysztof Czarnecki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rick Salay .

Editor information

Editors and Affiliations

  1. Newcastle University, Newcastle-upon-Tyne, UK

    Alexander Romanovsky

  2. Åbo Akademi University, Turku, Finland

    Elena Troubitsyna

  3. City, University of London, London, UK

    Ilir Gashi

  4. AIT Austrian Institute of Technology, Vienna, Austria

    Erwin Schoitsch

  5. Thales Deutschland GmbH, Berlin, Germany

    Friedemann Bitsch

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Salay, R., Czarnecki, K. (2019). Improving ML Safety with Partial Specifications. In: Romanovsky, A., Troubitsyna, E., Gashi, I., Schoitsch, E., Bitsch, F. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2019. Lecture Notes in Computer Science(), vol 11699. Springer, Cham. https://doi.org/10.1007/978-3-030-26250-1_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-26250-1_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-26249-5

  • Online ISBN: 978-3-030-26250-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation