Incorporating an ethical perspective into problem formulation: implications for decision support systems design

Abstract

As organizations become ever larger and increasingly complex, they become more reliant on information systems and decision support systems (DSS), and their decisions and operations affect a growing number of stakeholders. This paper argues that DSS design and problem formulation in such a context raises ethical issues, as DSS development and use puts one party, the designers, in the position of imposing order on the behavior of others. Thus, decision support systems are more than technical artifacts and their implications for affected parties should be considered in their design and development. The paper integrates Jones' model [Acad. Manage. Rev. 16 (1991) 366] of moral intensity with Mitroff's five strategies for avoiding Type III errors [I.I. Mitroff, Smart Thinking for Crazy Times: The Art of Solving the Right Problems, Barrett-Koehler Publishers, San Francisco, 1997], solving the wrong problem [H. Raiffa, Decision Analysis, Addison-Wesley, Reading, 1968], and proposes a model for incorporating ethical issues into DSS design and problem formulation. A survey of managers is used to assess the current situation regarding use of elements of the integrated model. The results are somewhat encouraging in that 40% of the respondents felt that their organizations did follow the model reasonably well, yet 23% felt their organizations did not.

Introduction

As organizations become ever larger and increasingly complex, they become more reliant on information systems and decision support systems (DSS), and their decisions and operations affect a growing number of stakeholders. For example, it was reported [5], [34], [64] that, on October 27, 1992, the city of London installed a new computer system for dispatching ambulances. Within a few hours, the system was overwhelmed by call volume and, the next day, the media reported that many lives had been lost due to the failure of ambulances to report where needed. The software house that developed the system had little expertise in the field and the system was technically inadequate to handle even ordinary call volumes. Moreover, the system had been built in a hostile environment between management and dispatchers, and users were not involved in the design process. Dispatchers may have sabotaged the system by giving it false information. This is a dramatic illustration of how dependence on computer systems can affect our lives, and how unethical behavior in the development of a system on the part of many involved led to a tragic outcome.

Today, a new DSS called the Computer Assisted Passenger Prescreening System (CAPPS II) illustrates this point well. CAPPS II is a nationwide computer system based on neural-network-based predictive software. The U.S. Congress ordered the system after the September 11, 2000 attacks and it is in development now. The system is designed to check such things as credit reports, consumer transactions, travel history and demographic information, to monitor passenger profiles and to generate a threat index or score for every passenger. Passengers will be asked for names, addresses and their date of birth before being allowed to board the plane. The information that passengers give will be used to create credit reports on passengers and to compare their names with government watch lists. Critics see a potential for invasions of privacy, for the likelihood of incorrect or biased information in a person's profile, for mismanagement of database data, for misidentifications by neural-network profiling, for mass surveillance and other problems (e.g., [3], [39]).

Leveraging data and information to such a great extent and in such a timely manner would be impossible without the use of modern DSS. However, the technology that makes these manipulations possible also divorces the person represented by the data from the decision-making perspective of the DSS user. Introna [25] notes that DSS in these situations impart a hyperreality for decision-makers and makes “ethical sensibility nebulous” to the point that DSS users no longer imagine the faces of those affected by decisions made using the system. Thus, the DSS users never come face-to-face with important stakeholders that may be affected by decisions based on the system's outputs.

Building on work by Mitroff and Linstone [37] and observing that decision-making processes focus on increasingly complex contexts, Courtney [13] argues that a new paradigm for decision-making is needed within decision support systems. Rather than going directly into analysis (a technical perspective) in a decision-making situation, he recommends a process that develops multiple perspectives (see Fig. 1). The various perspectives provide much greater insight into the nature of the problem and its possible solutions, than the heavy reliance on the technical perspective that DSS has advocated in the past. He argues that the missing piece in the existing DSS paradigm is consideration for broad organizational and personal perspectives, as well as ethical and aesthetic issues. What is missing in Courtney's work is some explanation of how the non-technical perspectives, in particular the components intended to incorporate ethical and aesthetical decision-making concerns, actually would be implemented. This paper focuses on support for incorporating an ethical perspective in decision support processes. The aesthetic perspective is addressed in Paradice [41].