Abstract
Current certification procedures aim at establishing trust in manufacturers of artificial intelligence/machine learning based cyber-physical systems. The certification process usually requires the manufacturer to demonstrate excellence in following safety engineering standards and regulations throughout the holistic system’s engineering process. This paper touches on the need for real-world performance monitoring performed by the certifier to ensure that the operational system does not deviate from the specifications. We propose an interactive cooperative process between the manufacturer and certifier which aims at verifying conformance and consistency between the specifications and the operational model while preserving the manufacturer’s competitive advantage.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
An (AI/ML)-based device is a system designed using AI and ML techniques to learn from and act on data [6].
References
Banerjee, A., Lamrani, I., Gupta, S.K.: Faultex: explaining operational changes in terms of design variables in cps control code. In: Proceeding of the 4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS). IEEE Press (2021)
Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.: Neural ordinary differential equations. arXiv preprint arXiv:1806.07366 (2018)
Fan, C., Qi, B., Mitra, S., Viswanathan, M.: DryVR: data-driven verification and compositional reasoning for automotive systems. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 441–461. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_22
Fan, C., Qi, B., Mitra, S., Viswanathan, M., Duggirala, P.S.: Automatic reachability analysis for nonlinear hybrid models with C2E2. In: Chaudhuri, S., Farzan, A. (eds.) CAV 2016. LNCS, vol. 9779, pp. 531–538. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41528-4_29
FDA: Developing a software precertification program: a working model (2019)
FDA: Proposed regulatory framework for modifications to artificial intelligence/machine learning (AI/ML)-based software as a medical device (2019)
Henzinger, T.A.: The theory of hybrid automata. In: Inan, M.K., Kurshan, R.P. (eds.)Verification of Digital and Hybrid Systems, NATO ASI Series (Series F: Computer and Systems Sciences), pp. 265–292. Springer, Heidelberg (2000). https://doi.org/10.1007/978-3-642-59615-5_13
Henzinger, T.A., Ho, P.H., Wong-Toi, H.: Hytech: a model checker for hybrid systems. Int. J. Softw. Tools Technol. Transf. (1997)
Kanderian Jr, S.S., Steil, G.M.: Apparatus and method for controlling insulin infusion with state variable feedback, 5 October 2010. US Patent 7,806,886
Koopman, P.: A case study of Toyota unintended acceleration and software safety. Presentation (2014)
Koopman, P., Ferrell, U., Fratrik, F., Wagner, M.: A safety standard approach for fully autonomous vehicles. In: Romanovsky, A., Troubitsyna, E., Gashi, I., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2019. LNCS, vol. 11699, pp. 326–332. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26250-1_26
Koopman, P., Fratrik, F.: How many operational design domains, objects, and events? SafeAI@ AAAI 4 (2019)
Lamrani, I., Banerjee, A., Gupta, S.K.: Hymn: mining linear hybrid automata from input output traces of cyber-physical systems. In: 2018 IEEE Industrial Cyber-Physical Systems (ICPS), pp. 264–269. IEEE (2018)
Lamrani, I., Banerjee, A., Gupta, S.K.: Operational data driven feedback for safety evaluation of agent-based cps. IEEE Trans. Ind. Inform. (2020)
Medhat, R., Ramesh, S., Bonakdarpour, B., Fischmeister, S.: A framework for mining hybrid automata from input/output traces. In: Proceedings of the 12th International Conference on Embedded Software, pp. 177–186. IEEE Press (2015)
Summerville, A., Osborn, J., Mateas, M.: Charda: causal hybrid automata recovery via dynamic analysis. arXiv preprint arXiv:1707.03336 (2017)
Acknowledgment
This work is partly funded by a DARPA AMP grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lamrani, I., Banerjee, A., Gupta, S.K.S. (2021). Certification Game for the Safety Analysis of AI-Based CPS. In: Habli, I., Sujan, M., Gerasimou, S., Schoitsch, E., Bitsch, F. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2021 Workshops. SAFECOMP 2021. Lecture Notes in Computer Science(), vol 12853. Springer, Cham. https://doi.org/10.1007/978-3-030-83906-2_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-83906-2_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-83905-5
Online ISBN: 978-3-030-83906-2
eBook Packages: Computer ScienceComputer Science (R0)