Decision making under time pressure with different information sources and performance-based financial incentives—Part 1
Introduction
Consider the following scenarios.
(a) Arnold, a stockbroker, must assess and classify stocks based on a number of factors. He monitors specific information about stocks of interest using an electronic monitoring device and makes rapid sell/buy/hold decisions.
(b) You are at home alone when you suspect a prowler. You press a special key on your telephone and a preformatted message “capsule” is routed to a 911 station. There, an operator receives and decodes the capsule and initiates appropriate steps, in real time.
(c) Virginia is a service specialist with DaimlerChrysler. When something goes wrong with a customer's late model Mercedes sedan on an important trip, the on-board computer sends short, precisely coded diagnostic information to Virginia's small hand-held/pocket device. The device alerts Virginia who decodes the received messages, enabling relatively quick diagnoses of what could be wrong with the vehicle. She is better prepared for all likely possibilities when she arrives at the scene for repairs within the hour.
(d) Jack, a hazardous materials expert, monitors packages on a conveyor system with the help of an expert system. The system scans each package and sends a report to the human decision maker. Based on this information, Jack makes a quick decision on how to deal with each package, without actually seeing or handling any of them.
Each of the above scenarios depicts a situation that conveys some urgency and involves “real-time” activity leading to categorizing or slotting the situation in some way during the course of making decisions. Some of the scenarios are realistic, others futuristic, even imaginary. In all of these examples, however, information concerning the situation must be conveyed quickly, yet precisely and unambiguously. Delays or errors in receiving a message, or in processing a received message, could have potentially serious consequences. Each represents a situation where speed and accuracy are both important.
Confusion is inherent in crisis situations. When a rape or knife-attack victim dials 911, she is in no condition to articulate her plea for help in the most “efficient and effective” way, especially if the perpetrator is suspected to be still in the vicinity. Yet, the nation's aging 911 system continues to fully rely on audio phone calls to initiate its response. Improperly trained, inexperienced, overwhelmed, and, sometimes, callous operators tend to compound an already-difficult situation. Instead, a system where a precanned message is dispatched from a special device (perhaps, bundled with a telephone) could possibly help cut down on the likelihood of errors and delays in conveying/comprehending what occurred, to whom, and where. The hazardous materials expert must make quick, accurate decisions. If he takes an undue amount of time with one package, it or other packages awaiting examination may detonate. Yet, if he makes a mistake with a package, this again could result in equally serious repercussions. A system equipped to convey essential features of a package using a precise, predetermined vocabulary alleviates such costly risks.
It is desirable to keep message length short and precise for two reasons. First, a short, precise message helps quickly convey what needs to be conveyed by reducing communication delays and the likelihood of transmission and/or imprecision-induced human errors. Second, despite their plummeting prices, functionally versatile, electronic computing machines like the seemingly ubiquitous desktop and laptop PCs are seemingly giving way to smaller, lighter, and more mobile functionally “narrow” units, such as pagers, web phones and PDAs. This movement is being fueled by the communication revolution that has superseded the computing revolution of the 70s, 80s, and early 90s. The smaller and lighter the device, the more portable it is, but the less memory and display it usually accommodates. Of special interest to us are devices that are intended for handling short, “bursty” traffic (as in our example scenarios) rather than sustained, high-bandwidth communications (e.g., the contents of a book or a videocassette).
Our investigation occurs within a context of time-pressured, performance-based incentive-driven settings, very similar to those faced by the decision makers in the example scenarios described above. We examine the performance of subjects when utilizing different kinds of information presentation “modes.” In particular, we study the efficacy of a new, symbolic communication mode that we call “Image” (and which is distinct from the familiar “icons”) vis-à-vis traditional written and spoken English which we call “Text” and “Audio,” respectively.
Given the small display of the decision support devices of interest, written English presents at least two disadvantages. First, it must be presented and understood linearly and sequentially, from left to right. This imposes time delays in both presentation and comprehension. Second, if the message being conveyed is not succinct enough for the display, then one must resort to moving (i.e., in “leading” form) displays or fragmented presentation. Both approaches have the disadvantage that the message is not available in its entirety or in substantial-enough chunks for sustained viewing. Spoken English has all of the disadvantages of a moving textual message (perhaps in exacerbated form) with the added drawback of offering no visual support.
Despite such drawbacks, traditional written and spoken communications dominate human–human interaction and, the former, much of human–machine interaction. The symbolic mode that we study is intended as one alternative, in the particular classification decision setting considered, to alleviate the drawbacks of the more traditional approaches. Yet, it is one that is entirely unfamiliar to new users, such as our subjects. Given identical experimental conditions for all modes, our subjects would have to comprehend and apply the novel Image mode within the same time and other restrictions made available to them when using the considerably more familiar Text and Audio. If our subjects performed just as well, or better, with the new language, then we would have a basis for further exploration of this and other such languages.
For ease of exposition, we report this study in two parts. Part 1 (the current paper) provides the motivation and background for the work reported in Part 2. Part 2 [28] describes the design, execution, and findings of an exploratory study of subject ability to recall and apply decision support information presented using each of the three communication modes—Text, Audio, and Image—in the presence of induced time pressure and performance-based financial incentives.
The remainder of Part 1 is organized as follows. In Section 2, we present a taxonomy and review of relevant literature. In Section 3, we describe key distinguishing features of our research vis-à-vis prior media-related and multimedia systems-related studies. Section 4 is concerned with the three kinds of information made available to a subject during experimentation: that concerning classification rules, sample objects, and to help make a final decision. The section concludes with a discussion on a product called the Training Tool Set developed for training our subjects. In order to make sure that the generic features and experiment-specific settings of the platform and tool set were adequate, we had these critiqued by a diverse group of individuals. Section 5 describes this effort. Section 6 contains concluding remarks.
Section snippets
A taxonomy and review of relevant literature
This section organizes and reviews relevant past literature. For convenience, we group this literature into the six categories shown in Table 1. Categories 1, 2, 3, 4a, and 4b, are concerned with key aspects of our study—i.e., task performance under time constraints, the use of laboratory experimentation, induced value theory, and the ex-ante evaluation of systems and alternate media—and we restrict our review to these. In each case, we focus on experimental conditions, rather than findings, as
Distinguishing features of this effort
This work differs from other media-related and multimedia systems-related studies in three important respects.
First, many prior studies (e.g., Refs. [3], [8], [9]) focus on the concept of “time horizon,” but not on “time pressure.” A time horizon is essentially a limit on an experiment's duration. Pressure is induced when: (1) subjects are encouraged, through suitable incentive mechanisms, to not merely meet a time limit, but to beat it, perhaps by as much as they can; or (2) subjects must
Prototype system characteristics and features
The remainder of Part 1 focuses on the design, development, and refinement of the experimental platform and related software that we believe are major contributions of this research. We created the platform and related software using the Asymetrix Multimedia ToolBook Version 3.0 package and deployed these on five Gateway 2000 P5-120 PCs running Windows 3.1 that were part of the erstwhile MIS Research Lab of the C.M. Gatton College of Business and Economics, University of Kentucky. Each station
Prototype assessment and refinement
Prototype module assessment and refinement were done with the assistance of 31 volunteers. The group included nine doctoral, six masters, and ten undergraduate students as well as six Gatton College of Business and Economics, University of Kentucky, employees, including the MIS manager. The volunteers were brought into the lab at their convenience over a three-week period and asked to voice their opinions concerning the experimental environment, the Training Tool Set modules, and the actual
Concluding remarks
In Part 1 of this two-part paper, we have:
(a) provided a strong motivation for this work by drawing on current technology trends and a review of the literature related to decision making under time pressure, induced value theory, ex-ante DSS evaluation, and ex-ante media evaluation;
(b) described how we constructed a prototype experimental platform to illustrate our ideas and an associated subject training application; and
(c) discussed how we assessed and refined these products to yield their
Acknowledgements
The authors express their sincere thanks to the three anonymous reviewers for their critical comments that enabled significant improvements to this two-part manuscript.
Dr. James R. Marsden, the Shenkman Family Chair in e-Business, came to UConn in 1993 as Professor and Head, Department of Operations and Information Management, School of Business Administration, University of Connecticut. Dr. Marsden was part of a three-person concept development team that initiated and oversaw the development of the Connecticut Information Technology Institute and is currently serving as its Executive Director. He developed and implemented the Treibick Electronic Commerce
References (50)
On integrating hypermedia into decision support and other information systems
Decision Support Systems
(1995)Factors that affect user-friendliness in interactive computer programs
Information and Management
(1992)- et al.
The design and use of laboratory experiments for DSS evaluation
Decision Support Systems
(1993) Decision making under time pressure: a model for information systems research
Information and Management
(1994)- et al.
Impact of design methods on decision support systems success: an empirical assessment
Information and Management
(1985) - et al.
The case for speech synthesis: an experiment in human engineering
Information and Management
(1988) - et al.
Determinants of the effectiveness of personal decision support systems
Information and Management
(1986) - M. Aminilari, Searching for Information: Experiences with a Text-based and an Image-based Decision Support System,...
- et al.
Measuring user participation, user involvement, and user attitude
MIS Quarterly
(1994) - et al.
An investigation of the effectiveness of color and graphical information presentation under varying time constraints
MIS Quarterly
(1986)
On generalizing the concept of hypertext
MIS Quarterly
HyperIntelligence: the next frontier
Communications of the ACM
Modeling the cognitive content of display
Human Factors
Factors affecting the readability of moving text on a computer display
Human Factors
The measurement of end-user computing satisfaction
MIS Quarterly
Visual display: the highlighting paradox
Human Factors
Minimizing the time to search visual displays: the role of highlighting
Human Factors
Expert systems and multimedia: examining the potential for integration
Journal of Management Information Systems
DSS evaluation: a comparison of ex-ante and ex-post evaluation methods
Hypermedia design, analysis, and evaluation issues
Communications of the ACM
Reflections on notecards: seven issues for the next generation of hypermedia systems
Communications of the ACM
Experimental law and economics: an introduction
Columbia Law Review
Computer-based decision aids: the evaluation problem
Sloan Management Review
Adaptive design for decision support systems
Database
Value analysis: justifying decision support systems
MIS Quarterly
Cited by (0)
Dr. James R. Marsden, the Shenkman Family Chair in e-Business, came to UConn in 1993 as Professor and Head, Department of Operations and Information Management, School of Business Administration, University of Connecticut. Dr. Marsden was part of a three-person concept development team that initiated and oversaw the development of the Connecticut Information Technology Institute and is currently serving as its Executive Director. He developed and implemented the Treibick Electronic Commerce Initiative that is funded through a generous gift provided by Richard Treibick and the Treibick Family Foundation. Dr. Marsden also serves as Director of the OPIM/SBA MIS Research Lab and is a member of Advisory Board and Steering Committee of CIBER (Center for International Business Education and Research). He was a member of the edgelab development team and currently serves on the edgelab Steering Committee which selects and resources projects and oversees operations. Dr. Marsden was a winner of the initial Chancellor's Award for IT Excellence and has a lengthy record in market innovation and analyses, economics of information, artificial intelligence, and production theory. His research work has appeared in Management Science; IEEE Transactions on Systems, Man, and Cybernetics; American Economic Review; Journal of Economic Theory; Journal of Political Economy; Computer Integrated Manufacturing Systems; Decision Support Systems; Journal of Management Information Systems, and numerous other academic journals. He was part of the IT Visioning and IT Planning Groups for the University and has played a leading role in developing the School of Business Administration as both a campus and national leader in IT education and research.
Professor Marsden received his AB (Phi Beta Kappa, James Scholar, Evans Scholar) degree from the University of Illinois and his MSc and PhD degrees from Purdue University. Having completed his J.D. while at the University of Kentucky, Jim has been admitted to both the Kentucky and Connecticut Bar. He is an Area Editor of Decision Support Systems and serves in a frequent external evaluator for major U.S. and international universities. He has held visiting positions at the University of York (England), University of Arizona, Purdue University, and the University of North Carolina. Jim was an Invited Lecturer at two NATO Advanced Study Institutes on Decision Support Systems and has given keynote addresses and university seminars throughout Europe and the Far East.
Ramakrishnan Pakath is an Associate Professor of Decision Science and Information Systems at the University of Kentucky. Ram holds an MSE (OR and IE) degree from The University of Texas at Austin and a PhD (Management-MIS) degree from Purdue University. His research focuses on (a) designing and evaluating adaptive problem processors, and (b) assessing information source impacts on system user performance. Dr. Pakath's research articles have appeared in such refereed forums as Decision Sciences, Decision Support Systems, European Journal of Operational Research, IEEE Transactions on Systems, Man, and Cybernetics, Information and Management, and Information Systems Research. He is author of the book Business Support Systems: An Introduction published by Copley, now in its second edition. Dr. Pakath has also contributed refereed material to a number of well-known books including Handbook of Industrial Engineering, Multimedia Technology and Applications, and Operations Research and Artificial Intelligence. He served as Director of the MIS Research Laboratory of the College of Business and Economics, University of Kentucky from 1993 to 1997. He is an Associate Editor for Decision Support Systems and an Editorial Board Member of Journal of End User Computing and Management.
Kustim Wibowo is an Associate Professor in the MIS and Decision Sciences Department of the Eberly College of Business and IT, Indiana University of Pennsylvania. He received his PhD in MIS from the University of Kentucky. Dr. Wibowo also holds an MSc in Computer Science from Baylor University. His current research interests include: e-commerce and web security, information systems for educational technology, human resource information systems, and OLAP (OnLine Analytical Processing) for managerial decision support.
- 1
This author's work was supported by the Treibick Electronic Commerce Initiative at the University of Connecticut.
- 2
Their work was supported by a grant from the MIS Research Laboratory Endowment at the University of Kentucky.
- 3
Presently at MIS and Decision Sciences, Eberly College of Business, Indiana University of Pennsylvania, Indiana, PA 15701.