Responsibility and Explainability in Using Intelligent Systems

Abstract

Recent developments in digital technology domains like Artificial Intelligence and Internet of Things lead to possibilities for more autonomous systems that more directly interact with their business or social environment. This poses new but often implicit questions with respect to the responsibility for decisions made by these systems and the effects of these decisions on their environment. Requirements embedded in design considerations like security constraints, privacy regulations, and traceability demands, however, imply explicit questions with respect to responsibility in information processing. Responsibility may be expected from four different classes of actors: the system itself, the system designer, the system user, and the party whose data is managed by the system, i.e., the object of the system. This paper presents the questions regarding the allocation of responsibility with the development and deployment of autonomous systems, based on requirements imposed by their application context. The discussion of the effect of explainability of decisions on the allocation of responsibility is an essential ingredient here. We provide the necessary concepts and models to make a first analysis and design of the allocation of responsibility.