Abstract
Machine Learning (ML) is increasingly deployed in complex application domains replacing human-decision making. While ML has been surprisingly successful, there are fundamental concerns in wide scale deployment without humans in the loop. A critical question is the trustworthiness of such ML systems. Although there is research towards making ML systems more trustworthy, there remain many challenges. In this position paper, we discuss the challenges and limitations of current proposals. We focus on a more adversarial approach, borrowing ideas from certification of security software with the Common Criteria. While it is unclear how to get strong trustworthy assurances for ML systems, we believe this approach can further increase the level of trust.
This research was supported by Google under the “Democratizing AI and Building Trust In the Technology” project and grant R-726-000-006-646.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The Common Criteria is a well defined process and standard (ISO/IEC 15408) for evaluation of the functional and assurance requirements of IT/security software.
- 2.
The U.S. House Committee investigation into the Boeing 737 Max flight control system found weaknesses given compliance processes were delegated to Boeing [1].
- 3.
In CC, there is further step, the evaluator’s work is sent to another trusted third party, the certification body, to give an official certificate for the evaluator’s work.
- 4.
An attack model is needed to capture potential malicious users of the ML system who try to exploit the system.
References
https://transportation.house.gov/committee-activity/boeing-737-max-investigation (2020)
Arnold, M., et al.: FactSheets: increasing trust in AI services through supplier’s declarations of conformity. IBM J. Res. Dev. 63(4/5), 1–13 (2019)
Athalye, A., Carlini, N., Wagner, D.A.: Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples. In: International Conference on Machine Learning (2018)
Bender, E.M., Friedman, B.: Data statements for natural language processing: toward mitigating system bias and enabling better science. Trans. Assoc. Comput. Linguist. 6, 587–604 (2018)
Carlini, N., et al.: On evaluating adversarial robustness. https://arxiv.org/abs/1902.06705 (2019)
Dwork, C., Roth, A.: The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9(3–4), 211–407 (2014)
Friedler, S.A., Scheidegger, C., Venkatasubramanian, S.: On the (im)possibility of fairness (2016). http://arxiv.org/abs/1609.07236
Gebru, T., et al.: Datasheets for datasets. In: Workshop on Fairness, Accountability, and Transparency in Machine Learning (2018)
Holland, S., Hosny, A., Newman, S., Joseph, J., Chmielinski, K.: The dataset nutrition label: a framework to drive higher data quality standards. https://arxiv.org/abs/1805.03677 (2018)
Kifer, D., Machanavajjhala, A.: No free lunch in data privacy. In: ACM SIGMOD International Conference on Management of Data (2011)
Klein, G., et al.: Comprehensive formal verification of an OS microkernel. ACM Trans. Comput. Syst. 32(1), 1–70 (2014)
Kumar, V.: Irrational Exuberance and the ‘FATE’ of Technology. BLOG@CACM, 20 August (2018)
Mitchell, M., et al.: Model cards for model reporting. In: Conference on Fairness, Accountability, and Transparency (2019)
Obermeyer, Z., Powers, B., Vogeli, C., Mullainathan, S.: Dissecting racial bias in an algorithm used to manage the health of populations. Science 366(6464), 447–453 (2019)
Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550, 354–359 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yap, R.H.C. (2021). Towards Certifying Trustworthy Machine Learning Systems. In: Heintz, F., Milano, M., O'Sullivan, B. (eds) Trustworthy AI - Integrating Learning, Optimization and Reasoning. TAILOR 2020. Lecture Notes in Computer Science(), vol 12641. Springer, Cham. https://doi.org/10.1007/978-3-030-73959-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-73959-1_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73958-4
Online ISBN: 978-3-030-73959-1
eBook Packages: Computer ScienceComputer Science (R0)