Abstract
Machine learning is a popular tool for building state of the art software systems. It is more and more used also in safety critical areas. This demands for verification techniques ensuring the safety and security of machine learning based solutions. However, we argue that the popularity of machine learning comes from the fact that no formal specification exists which renders traditional verification inappropriate. Instead, validation is typically demanded, but formalization of so far informal requirements is necessary to give formal evidence that the right system is build. Moreover, we present a recent technique that allows to check certain properties for an underlying recurrent neural network and which may be uses as a tool to identify whether the system learned is right.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
“In short, Boehm (3) expressed the difference between the software verification and software validation as follows: Verification: “Are we building the product right?” Validation: “Are we building the right product?” [12].
- 2.
- 3.
The requirements are listed here in a very brief form. Please consult the original article for an elaborate explanation.
References
Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987). https://doi.org/10.1016/0890-5401(87)90052-6
Baier, C., Katoen, J.: Principles of Model Checking. MIT Press, Cambridge (2008)
Bauer, A., Leucker, M.: The theory and practice of SALT. In: Bobaru, M., Havelund, K., Holzmann, G.J., Joshi, R. (eds.) NFM 2011. LNCS, vol. 6617, pp. 13–40. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20398-5_3
Deng, L., Hinton, G., Kingsbury, B.: New types of deep neural network learning for speech recognition and related applications: an overview. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8599–8603 (2013)
Dwyer, M.B., Avrunin, G.S., Corbett, J.C.: Patterns in property specifications for finite-state verification. In: Boehm, B.W., Garlan, D., Kramer, J. (eds.) International Conference on Software Engineering, ICSE 1999, pp. 411–420. ACM (1999)
Garavel, H., Beek, M.H., Pol, J.: The 2020 expert survey on formal methods. In: ter Beek, M.H., Ničković, D. (eds.) FMICS 2020. LNCS, vol. 12327, pp. 3–69. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58298-2_1
Gulwani, S., Polozov, O., Singh, R.: Program synthesis. Found. Trends®in Program. Lang. 4(1–2), 1–119 (2017). https://doi.org/10.1561/2500000010
High-Level Expert Group on AI: Ethics guidelines for trustworthy AI. Report, European Commission, Brussels, April 2019. https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai
Khmelnitsky, I., et al.: Property-directed verification of recurrent neural networks (2020). Preprint https://arxiv.org/abs/2009.10610
Liu, J., et al.: Applications of deep learning to MRI images: a survey. Big Data Min. Anal. 1(1), 1–18 (2018)
Mahmud, M., Kaiser, M.S., Hussain, A., Vassanelli, S.: Applications of deep learning and reinforcement learning to biological data. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2063–2079 (2018)
Kopetz, H.: Automatic Error Detection. Software Reliability, pp. 68–80. Macmillan Education UK, London (1976). https://doi.org/10.1007/978-1-349-86129-3_9
Pnueli, A.: The temporal logic of programs. In: Proceedings of the 18th IEEE Symposium on the Foundations of Computer Science (FOCS 1977), Providence, Rhode Island, 31 October–2 November 1977, pp. 46–57. IEEE Computer Society Press (1977)
Sejnowski, T.J.: The unreasonable effectiveness of deep learning in artificial intelligence. CoRR abs/2002.04806 (2020). https://arxiv.org/abs/2002.04806
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Leucker, M. (2020). Formal Verification of Neural Networks?. In: Carvalho, G., Stolz, V. (eds) Formal Methods: Foundations and Applications. SBMF 2020. Lecture Notes in Computer Science(), vol 12475. Springer, Cham. https://doi.org/10.1007/978-3-030-63882-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-63882-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63881-8
Online ISBN: 978-3-030-63882-5
eBook Packages: Computer ScienceComputer Science (R0)