Abstract
Normative monitoring of black-box AI systems entails detecting whether input-output combinations of AI systems are acceptable in specific contexts. To this end, we build on an existing approach that uses Bayesian networks and a tailored conflict measure called IOconfl. In this paper, we argue that the default fixed threshold associated with this measure is not necessarily suitable for the purpose of normative monitoring. We subsequently study the bounds imposed on the measure by the normative setting and, based upon our analyses, propose a dynamic threshold that depends on the context in which the AI system is applied. Finally, we show the measure and threshold are effective by experimentally evaluating them using an existing Bayesian network.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Even without including context, there can be various reasons why \(o^*\) need not be the most likely value of O given \(\textbf{i}\) in \({\Pr }^N\), for one thing because the normative model is not designed to make predictions regarding the value of O.
- 2.
Available from https://www.bnlearn.com/bnrepository/discrete-medium.html.
- 3.
The experiment was executed using the GeNIe Modeler and the SMILE Engine by BayesFusion, LLC (http://www.bayesfusion.com/).
References
Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 1–58 (2009)
Jensen, F.V., Chamberlain, B., Nordahl, T., Jensen, F.: Analysis in HUGIN of data conflict. In: Proceedings of the Sixth Conference on Uncertainty in Artificial Intelligence, pp. 546–554 (1990)
Jensen, F.V., Nielsen, T.D.: Bayesian Networks and Decision Graphs, 2nd edn. Springer, New York (2007). https://doi.org/10.1007/978-0-387-68282-2
Johansson, F., Falkman, G.: Detection of vessel anomalies - a Bayesian network approach. In: Proceedings of the Third International Conference on Intelligent Sensors, Sensor Networks and Information, pp. 395–400. IEEE (2007)
Kirk, A., Legg, J., El-Mahassni, E.: Anomaly detection and attribution using Bayesian networks. Technical report, Defence Science and Technology Organisation Canberra (2014)
Kjaerulff, U.B., Madsen, A.L.: Bayesian Networks and Influence Diagrams: A Guide to Construction and Analysis, 2nd edn. Springer, New York (2013). https://doi.org/10.1007/978-1-4614-5104-4
Nielsen, T.D., Jensen, F.V.: On-line alert systems for production plants: a conflict based approach. Int. J. Approximate Reasoning 45, 255–270 (2007)
Onnes, A.: Monitoring AI systems: a problem analysis, framework and outlook. In: Proceedings of the First International Conference on Hybrid-Artificial Intelligence. Frontiers in Artificial Intelligence and Applications, vol. 354, pp. 238–240 (2022)
Onnes, A., Dastani, M., Renooij, S.: Bayesian network conflict detection for normative monitoring of black-box systems. In: Proceedings of the Thirty-Sixth FLAIRS Conference, vol. 36. Florida Online Journals (2023)
Spiegelhalter, D.J., Dawid, A.P., Lauritzen, S.L., Cowell, R.G.: Bayesian analysis in expert systems. Statist. Sci. 8, 219–247 (1993)
Acknowledgements
This research was supported by the Hybrid Intelligence Centre, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research, https://hybrid-intelligence-centre.nl.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Onnes, A., Dastani, M., Renooij, S. (2024). Normative Monitoring Using Bayesian Networks: Defining a Threshold for Conflict Detection. In: Bouraoui, Z., Vesic, S. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2023. Lecture Notes in Computer Science(), vol 14294. Springer, Cham. https://doi.org/10.1007/978-3-031-45608-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-45608-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45607-7
Online ISBN: 978-3-031-45608-4
eBook Packages: Computer ScienceComputer Science (R0)