A process model cognitive biasing effects in information systems development and usage
Introduction
Under conditions of uncertainty, a decision maker (DM) is likely to search for available information from a vast array of sources and to accept information which conforms to some perception of a ‘reasonable’ outcome. Decision aids, such as decision support systems, are intended to reduce uncertainty and promote effective, objective solutions. Nonetheless, the retrieval and processing of information may be limited or distorted by the DM’s inherent cognitive biases. Further, given the involvement of a systems designers with their own set of cognitive biases, biased outcomes may actually be promoted since the information system was constructed based on the DM’s stated specifications.
Cognitive biases are neither readily apparent nor easily counterbalanced since they are developmentally inherent in the cognitive schemas and heuristics that form the basis of human information processing. These schemas and heuristics constitute behavioral patterns which dictate the way in which the DM approaches a problem. Such cognitive processes serve to reduce and combine mentally cumbersome quantities of information. They include filtering mechanisms applied in determining the elements or variables associated with the problem space, the analysis techniques used to examine the interrelationships between the components, and the manner in which outcomes are processed in relation to functional objectives.
While permitting the DM to manage complex information, numerous untoward consequences of cognitive heuristics and biases in decision making and information systems (IS) construction have been recognized by researchers for some time [4], [29], [34], [39], [42] (see [33] for a thorough review). More recently, the negative impacts of judgmental biases in day-to-day software development has been given attention by practitioners, especially with regard to testing and debugging [12], [37], [38].
Researchers generally agree that it would be futile to attempt to nullify biases, given their number, variety, and overlap with other biases, as well as the fact that they have can have positive utility in processing information. Furthermore, such efforts could possibly be counterproductive if they force the DM to alter successful patterns of behavior. However, in order to facilitate improved decision making, IS designers should understand the nature of such biases and the conditions under which they can be problematic. Counter-measures to alleviate some of the negative effects could subsequently be provided.
Some studies have offered suggestions for dealing with cognitive biases during IS design. These studies have focused on special problems of probabilistic information [8], [9], ways to incorporate control mechanisms to counter individual biases [4], [33], [37], or have focused on specific IS types [23], [29]. Although it has been suggested that a general model detailing the potential impacts of cognitive biases on IS design and usage is needed [4], [33], little effort has been directed toward this end.
The primary purpose of this paper is to consolidate and summarize the various biases that have been identified in various fields such as decision making, cognitive psychology and information systems. We are primarily concerned with biases which tend to arise in the development and use of small-scale IS. By that we mean those that are generally produced by a single designer, or under the direction of a single analyst, and are used by individual decision makers. This is in contrast to the larger, organization-wide systems which typically involve large teams of systems analysts to develop and are often used by teams of decision makers. While these latter systems can give rise to the same biases, because they are produced and used by groups of individuals, the situations which promote the biases, as well as how the biases manifest themselves, can vary.
It is not our intent to suggest that all biases are detrimental and should, therefore, be moderated. As shall be discussed, these biases underlie the cognitive schemas and heuristics which compose an IS designer’s and DM’s information processing patterns. Attempting to entirely alter the manner in which these individuals behave and reason is not only counterproductive, but also infeasible. However, dependent upon the available supply of information available, the information system derived from it, and the task environment, many of the biases can have detrimental affects. Our objective is to make the participants in this process, especially the IS designer, aware of the potential biases which exist, and what affect they could have on the decision making process. Consequently, the designer might be able to incorporate moderating strategies in developing the IS to alleviate some of unfavorable affects of biases (to be discussed later).
In Section 2, we review some conceptual foundations and past findings, drawing heavily from the psychology and IS literature. We next develop a process model to examine the manner in which the inherent behavioral schemas of the DM and the IS designer might filter or distort information during IS development, access, and application. A number of postulated biases are categorized in terms of their impact on the activities involved in the decision making process. In the Section 4, we discuss some of the theoretical constructs underlying cognitive processing mechanisms. Next, we propose a process model that illustrates some of the potential biases which may occur at various stages of the IS development and decision making process. We then suggest some preventive measures to alleviate some of the negative effects associated with these biases, and provide an example of how these measures might impact decision making. Finally, we offer some recommendations for future research.
Section snippets
Conceptual foundations and past findings
A number of theories have been advanced to describe human information processing. Most view cognitive processes as a set of information filtering and organizing mechanisms [6]. Because humans are inundated with enormous amounts of stimulus information, they tend to utilize a variety of mechanisms to cope with the environment. Coping mechanisms include chunking of information [34], [35], the use of heuristics [42], programmed strategies [18], and various other information categorization
A model of cognitive biases in information processing
The model, presented in Fig. 1, approaches the biases in information processing from two perspectives. It considers the two primary participants involved in development and usage of the information system, the IS designer and the DM (represented as shaded rectangles in the model), and three major components which impact the decision making process (represented as circles). These components include:
- 1.
The ‘Real-World’ universe of information, or Information supply, consisting of all facts and
Designer biases
Ideally, the IS designer should provide an objective analysis of the problem based on user requirements. In a large-scale, organization-wide IS, a team approach, relying on top–down systems analysis and design methodologies would likely be employed. As noted earlier, our focus is on smaller systems, where deployment of large systems analysis and design teams is not feasible.
For smaller systems, the IS designer can succumb to several biases. Many of these correspond to those noted for the DM
Decision maker biases
There are three linkages (labeled L3–L5 on Fig. 1) which may give way to a number of DM biases. Since these involve a variety of complex activities, the total number of potential biases postulated in considerably larger. The IS designer should be aware of the DM’s biases in order to construct an IS which moderate some of the negative impacts.
The linkages postulated for the DM are grouped according to the interface between the DM and the two major components of the information processing system
Moderation of potential biases
As mentioned previously, it is neither feasible nor desirable to attempt to eliminate all of the biases listed above. In some cases, however, it may desirable and possible to alleviate some of their potentially negative impacts.
There has been considerable research conducted, and prescriptions suggested, on techniques for avoiding biases, and di-biasing those that do occur [10], [16], [19], [21], [43]. Sage [33] has outlined the 10 most commonly mentioned strategies (see Table 2 for summary):
- 1.
Conclusion
This paper is intended to provide an overview of some of the IS designer and DM biases which can occur at various phases in the IS development and information processing cycle. The processing model presented is intended as a mechanism for categorizing bias occurrences according to the activities which take place in the information processing cycle.
Our main intent is to make the participants in the systems development and usage process, especially the IS Designer, aware of the potentially
References (47)
- et al.
Assessment of multiplicative utility functions vias holistic judgments
Organizational Behavior and Human Performance
(1980) - et al.
Probabilistic information processing systems: evaluation with conditionally dependent data
Organizational Behavior and Human Performance
(1972) - et al.
Comparison of Bayesian and regression approaches to the study of information processing in judgment
Organizational Behavior and Human Performance
(1971) Availability: a heuristic for judging frequency and probability
Cognitive Psychology
(1973)- R.P. Abelson, Psychological status of the script concept, American Psychologist,...
The base-rate fallacy in probability judgments
ACTA Psychology
(1980)- et al.
Behavioral Aspects of information processing for the design of management information systems
IEEE Transactions on Systems, Man and Cybernetics
(1982) - et al.
The differential impact of abstract vs. concrete information in decisions
Journal of Applied Social Psychology
(1977) The well-ordered mind
American Educational Research Journal
(1966)- J.S. Brunner, R. Oliver, P.M. Greenfield, Studies in Cognitive Growth, Wiley, New York,...
Probabilistic information processing systems: design and evaluation
IEEE Transactions in Systems, Science and Cybernetics (SSC)
Fault trees: sensitivity of estimated failure probabilities to problem presentation
Journal of Experimental Psychology: Human Perception and Performance
Forecasting and planning: an evaluation
Management Science
The effects of problem representation on the sure thing and substitution principles
Management Science
On the psychology of prediction
Psychology Review
Cited by (26)
People with jumping to conclusions bias tend to make context-independent decisions rather than context-dependent decisions
2022, Consciousness and CognitionCitation Excerpt :Cognitive biases have been observed in the general population. Due to information processing limitations, people need to use a variety of information simplification mechanisms that would help them process and interpret enormous amounts of data they encounter every day and take decisions under a wide range of complex, uncertain situations (Kirs et al., 2001). Although there are often biases in human judgment and decision-making, not all people exhibit these frequently dysfunctional information processing patterns.
Relations between Cognitive Biases and Some Concepts of Information Behavior
2020, Data and Information ManagementCitation Excerpt :In information science, while cognitive biases have not received much attention in the domain of human information behavior, there have been studies on relevant concepts such as uncertainty (e.g., works by Kuhlthau, 1993; Ingwersen, 1996; and Wilson, Ford, Ellis, Foster, & Spink, 2002), which plays an important role in information behavior. Cognitive biases have received more attention from researchers in many areas, such as information systems (e.g., Kirs, Pflughoeft, & Kroeck, 2001; Arnott, 2006) and health information (e.g., Lau & Coiera, 2007, 2009; Schweiger, Oeberst, & Cress, 2014). There are two reviews of research on cognitive biases in information systems (Fleischmann, Amirpur, Benlian, & Hess, 2014; Mohanani, Salman, Turhan, Rodriguez, & Ralph, 2017).
The effects of optimistic and pessimistic biasing on software project status reporting
2007, Information and ManagementUnravelling Behavioural Biases in Individual and Institutional Investors Investment Decision- making: Intersection of Bibliometric and Systematic Literature Review
2023, South Asian Journal of Business and Management CasesMisrepresentations and Criminal Liability in Project Reporting: A Case Study of the Failed Virgil C. Summer Project
2023, Journal of Legal Affairs and Dispute Resolution in Engineering and ConstructionComplied by Belief Consistency: The Cognitive-Information Lens of User-Generated Persuasion
2023, Journal of Theoretical and Applied Electronic Commerce Research