Abstract
We demonstrate the use of formal methods to verify properties of a deep convolutional neural network that estimates remaining useful life of aircraft mechanical equipment. We provide mathematical formalizations of requirements of the estimator, such as stability and monotonicity, as properties. To efficiently apply existing tools for verification of neural networks, we reduce the verification of global properties to a representative set of local properties defined for the points of the test dataset. We encode these properties as linear constraints and verify them using a state-of-the-art tool for neural network verification. To increase the completeness and the scalability of the analysis, we develop a two-step verification method involving abstract interpretation and simulation-based falsification. Numerical results confirm the applicability of the approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This is a domain-specific concept, different e.g. from the notion of stability in control theory; see Sect. 3.1 for details.
- 2.
In this case, the notion of monotonicity applies to a CI in a time window, intended as a sequence of values, rather than to the RUL w.r.t. one of the CIs.
- 3.
A more detailed investigation of the reduction of global to local properties would include producing empirical evidence of the feasibility of the approach, as well as deriving analytical bounds on the approximation. This is subject of future work.
References
Remaining Useful Life Estimation using Convolutional Neural Network. https://www.mathworks.com/help/releases/R2021a/predmaint/ug/remaining-useful-life-estimation-using-convolutional-neural-network.html
Similarity-Based Remaining Useful Life Estimation. https://www.mathworks.com/help/predmaint/ug/similarity-based-remaining-useful-life-estimation.html
Benkedjouh, T., Medjaher, K., Zerhouni, N., Rechak, S.: Remaining useful life estimation based on nonlinear feature reduction and support vector regression. Eng. Appl. Artif. Intell. 26, 1751–1760 (2013)
Damour, M., et al.: Towards certification of a reduced footprint ACAS-Xu system: a hybrid ML-based solution. In: Habli, I., Sujan, M., Bitsch, F. (eds.) SAFECOMP 2021. LNCS, vol. 12852, pp. 34–48. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-83903-1_3
EASA and Collins Aerospace: Formal Methods use for Learning Assurance (ForMuLA). Tech. rep. (April 2023)
European Union Aviation Safety Agency (EASA): Concept Paper: Guidance for Level 1 &2 Machine Learning Applications. Concept paper for consultation (February 2023)
Katz, G.: The marabou framework for verification and analysis of deep neural networks. In: Dillig, I., Tasiran, S. (eds.) CAV 2019. LNCS, vol. 11561, pp. 443–452. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-25540-4_26
Li, X., Ding, Q., Sun, J.Q.: Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Eng. System Safety, 1–11 (2018)
Liu, C., Cofer, D., Osipychev, D.: Verifying an aircraft collision avoidance neural network with marabou. In: Proceeding of NASA Formal Methods Symposium (2023)
Pecht, M., Gu, J.: Physics-of-failure-based prognostics for electronic products. IEEE Trans. Measurem. Control 31, 309–322 (2009)
Ren, L., Cui, J., Sun, Y., Cheng, X.: Multi-bearing remaining useful life collaborative prediction: a deep learning approach. J. Manuf. Syst. 43, 248–256 (2017)
RTCA/DO-178C: Software Considerations in Airborne Systems and Equipment Certification (2011)
RTCA/DO-333: Formal Methods Supplement to DO-178C and DO-278A (2011)
SAE G-34 Artificial Intelligence in Aviation: Artificial Intelligence in Aeronautical Systems: Statement of Concerns (2021)
Tran, H.-D., et al.: NNV: the neural network verification tool for deep neural networks and learning-enabled cyber-physical systems. In: Lahiri, S.K., Wang, C. (eds.) CAV 2020. LNCS, vol. 12224, pp. 3–17. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-53288-8_1
Wang, S., Pei, K., Whitehouse, J., Yang, J., Jana, S.: Efficient formal safety analysis of neural networks. In: Advances in Neural Information Processing Systems 31 (2018)
Acknowledgements
The authors wish to thank Eric DeWind and David F. Larsen for fruitful discussions and feedback about the RUL estimator, as well as for providing mechanical bearing degradation datasets to train and test the neural network.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kirov, D., Rollini, S.F., Di Guglielmo, L., Cofer, D. (2024). Formal Verification of a Neural Network Based Prognostics System for Aircraft Equipment. In: Steffen, B. (eds) Bridging the Gap Between AI and Reality. AISoLA 2023. Lecture Notes in Computer Science, vol 14380. Springer, Cham. https://doi.org/10.1007/978-3-031-46002-9_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-46002-9_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46001-2
Online ISBN: 978-3-031-46002-9
eBook Packages: Computer ScienceComputer Science (R0)