Elsevier

Decision Support Systems

Volume 34, Issue 1, December 2002, Pages 75-97
Decision Support Systems

Decision making under time pressure with different information sources and performance-based financial incentives—Part 1

https://doi.org/10.1016/S0167-9236(01)00135-XGet rights and content

Abstract

We are witness to the communications revolution and the accompanying proliferation of narrow-purpose, mobile, computing and communication devices. Such devices tend to be smaller and lighter than their desktop and laptop counterparts. The tradeoff is that their displays and memory also tend to be relatively smaller. To date, they also rely on traditional English and/or icons for communicating with users. While icons have grown in usage, capturing any and all information using icons is impossible and/or prohibitively expensive. We examine the viability of developing new kinds of communication languages for such devices in a specific setting by considering an abstract classification task and examining the performance of subjects using a new, compact language that we have devised vis-à-vis written and spoken English. Our work draws on prior research on induced value experimentation and ex-ante system evaluation. In Part 1 of this two-part paper, we provide the necessary background, discuss the underlying motivations, and describe the construction and refinement of our experimental platform and an accompanying subject training software suite.

Introduction

Consider the following scenarios.

(a) Arnold, a stockbroker, must assess and classify stocks based on a number of factors. He monitors specific information about stocks of interest using an electronic monitoring device and makes rapid sell/buy/hold decisions.

(b) You are at home alone when you suspect a prowler. You press a special key on your telephone and a preformatted message “capsule” is routed to a 911 station. There, an operator receives and decodes the capsule and initiates appropriate steps, in real time.

(c) Virginia is a service specialist with DaimlerChrysler. When something goes wrong with a customer's late model Mercedes sedan on an important trip, the on-board computer sends short, precisely coded diagnostic information to Virginia's small hand-held/pocket device. The device alerts Virginia who decodes the received messages, enabling relatively quick diagnoses of what could be wrong with the vehicle. She is better prepared for all likely possibilities when she arrives at the scene for repairs within the hour.

(d) Jack, a hazardous materials expert, monitors packages on a conveyor system with the help of an expert system. The system scans each package and sends a report to the human decision maker. Based on this information, Jack makes a quick decision on how to deal with each package, without actually seeing or handling any of them.

Each of the above scenarios depicts a situation that conveys some urgency and involves “real-time” activity leading to categorizing or slotting the situation in some way during the course of making decisions. Some of the scenarios are realistic, others futuristic, even imaginary. In all of these examples, however, information concerning the situation must be conveyed quickly, yet precisely and unambiguously. Delays or errors in receiving a message, or in processing a received message, could have potentially serious consequences. Each represents a situation where speed and accuracy are both important.

Confusion is inherent in crisis situations. When a rape or knife-attack victim dials 911, she is in no condition to articulate her plea for help in the most “efficient and effective” way, especially if the perpetrator is suspected to be still in the vicinity. Yet, the nation's aging 911 system continues to fully rely on audio phone calls to initiate its response. Improperly trained, inexperienced, overwhelmed, and, sometimes, callous operators tend to compound an already-difficult situation. Instead, a system where a precanned message is dispatched from a special device (perhaps, bundled with a telephone) could possibly help cut down on the likelihood of errors and delays in conveying/comprehending what occurred, to whom, and where. The hazardous materials expert must make quick, accurate decisions. If he takes an undue amount of time with one package, it or other packages awaiting examination may detonate. Yet, if he makes a mistake with a package, this again could result in equally serious repercussions. A system equipped to convey essential features of a package using a precise, predetermined vocabulary alleviates such costly risks.

It is desirable to keep message length short and precise for two reasons. First, a short, precise message helps quickly convey what needs to be conveyed by reducing communication delays and the likelihood of transmission and/or imprecision-induced human errors. Second, despite their plummeting prices, functionally versatile, electronic computing machines like the seemingly ubiquitous desktop and laptop PCs are seemingly giving way to smaller, lighter, and more mobile functionally “narrow” units, such as pagers, web phones and PDAs. This movement is being fueled by the communication revolution that has superseded the computing revolution of the 70s, 80s, and early 90s. The smaller and lighter the device, the more portable it is, but the less memory and display it usually accommodates. Of special interest to us are devices that are intended for handling short, “bursty” traffic (as in our example scenarios) rather than sustained, high-bandwidth communications (e.g., the contents of a book or a videocassette).

Our investigation occurs within a context of time-pressured, performance-based incentive-driven settings, very similar to those faced by the decision makers in the example scenarios described above. We examine the performance of subjects when utilizing different kinds of information presentation “modes.” In particular, we study the efficacy of a new, symbolic communication mode that we call “Image” (and which is distinct from the familiar “icons”) vis-à-vis traditional written and spoken English which we call “Text” and “Audio,” respectively.

Given the small display of the decision support devices of interest, written English presents at least two disadvantages. First, it must be presented and understood linearly and sequentially, from left to right. This imposes time delays in both presentation and comprehension. Second, if the message being conveyed is not succinct enough for the display, then one must resort to moving (i.e., in “leading” form) displays or fragmented presentation. Both approaches have the disadvantage that the message is not available in its entirety or in substantial-enough chunks for sustained viewing. Spoken English has all of the disadvantages of a moving textual message (perhaps in exacerbated form) with the added drawback of offering no visual support.

Despite such drawbacks, traditional written and spoken communications dominate human–human interaction and, the former, much of human–machine interaction. The symbolic mode that we study is intended as one alternative, in the particular classification decision setting considered, to alleviate the drawbacks of the more traditional approaches. Yet, it is one that is entirely unfamiliar to new users, such as our subjects. Given identical experimental conditions for all modes, our subjects would have to comprehend and apply the novel Image mode within the same time and other restrictions made available to them when using the considerably more familiar Text and Audio. If our subjects performed just as well, or better, with the new language, then we would have a basis for further exploration of this and other such languages.

For ease of exposition, we report this study in two parts. Part 1 (the current paper) provides the motivation and background for the work reported in Part 2. Part 2 [28] describes the design, execution, and findings of an exploratory study of subject ability to recall and apply decision support information presented using each of the three communication modes—Text, Audio, and Image—in the presence of induced time pressure and performance-based financial incentives.

The remainder of Part 1 is organized as follows. In Section 2, we present a taxonomy and review of relevant literature. In Section 3, we describe key distinguishing features of our research vis-à-vis prior media-related and multimedia systems-related studies. Section 4 is concerned with the three kinds of information made available to a subject during experimentation: that concerning classification rules, sample objects, and to help make a final decision. The section concludes with a discussion on a product called the Training Tool Set developed for training our subjects. In order to make sure that the generic features and experiment-specific settings of the platform and tool set were adequate, we had these critiqued by a diverse group of individuals. Section 5 describes this effort. Section 6 contains concluding remarks.

Section snippets

A taxonomy and review of relevant literature

This section organizes and reviews relevant past literature. For convenience, we group this literature into the six categories shown in Table 1. Categories 1, 2, 3, 4a, and 4b, are concerned with key aspects of our study—i.e., task performance under time constraints, the use of laboratory experimentation, induced value theory, and the ex-ante evaluation of systems and alternate media—and we restrict our review to these. In each case, we focus on experimental conditions, rather than findings, as

Distinguishing features of this effort

This work differs from other media-related and multimedia systems-related studies in three important respects.

First, many prior studies (e.g., Refs. [3], [8], [9]) focus on the concept of “time horizon,” but not on “time pressure.” A time horizon is essentially a limit on an experiment's duration. Pressure is induced when: (1) subjects are encouraged, through suitable incentive mechanisms, to not merely meet a time limit, but to beat it, perhaps by as much as they can; or (2) subjects must

Prototype system characteristics and features

The remainder of Part 1 focuses on the design, development, and refinement of the experimental platform and related software that we believe are major contributions of this research. We created the platform and related software using the Asymetrix Multimedia ToolBook Version 3.0 package and deployed these on five Gateway 2000 P5-120 PCs running Windows 3.1 that were part of the erstwhile MIS Research Lab of the C.M. Gatton College of Business and Economics, University of Kentucky. Each station

Prototype assessment and refinement

Prototype module assessment and refinement were done with the assistance of 31 volunteers. The group included nine doctoral, six masters, and ten undergraduate students as well as six Gatton College of Business and Economics, University of Kentucky, employees, including the MIS manager. The volunteers were brought into the lab at their convenience over a three-week period and asked to voice their opinions concerning the experimental environment, the Training Tool Set modules, and the actual

Concluding remarks

In Part 1 of this two-part paper, we have:

  • (a) provided a strong motivation for this work by drawing on current technology trends and a review of the literature related to decision making under time pressure, induced value theory, ex-ante DSS evaluation, and ex-ante media evaluation;

  • (b) described how we constructed a prototype experimental platform to illustrate our ideas and an associated subject training application; and

  • (c) discussed how we assessed and refined these products to yield their

Acknowledgements

The authors express their sincere thanks to the three anonymous reviewers for their critical comments that enabled significant improvements to this two-part manuscript.

Dr. James R. Marsden, the Shenkman Family Chair in e-Business, came to UConn in 1993 as Professor and Head, Department of Operations and Information Management, School of Business Administration, University of Connecticut. Dr. Marsden was part of a three-person concept development team that initiated and oversaw the development of the Connecticut Information Technology Institute and is currently serving as its Executive Director. He developed and implemented the Treibick Electronic Commerce

References (50)

  • M.P Bieber et al.

    On generalizing the concept of hypertext

    MIS Quarterly

    (1992)
  • D.A Carlson et al.

    HyperIntelligence: the next frontier

    Communications of the ACM

    (1990)
  • R.A Chechile et al.

    Modeling the cognitive content of display

    Human Factors

    (1989)
  • H Chen et al.

    Factors affecting the readability of moving text on a computer display

    Human Factors

    (1988)
  • W.J Doll et al.

    The measurement of end-user computing satisfaction

    MIS Quarterly

    (1988)
  • D.L Fisher et al.

    Visual display: the highlighting paradox

    Human Factors

    (1989)
  • D.L Fisher et al.

    Minimizing the time to search visual displays: the role of highlighting

    Human Factors

    (1989)
  • W.L Fuerst et al.

    Expert systems and multimedia: examining the potential for integration

    Journal of Management Information Systems

    (1995)
  • C Gardner et al.

    DSS evaluation: a comparison of ex-ante and ex-post evaluation methods

  • F Garzotto et al.

    Hypermedia design, analysis, and evaluation issues

    Communications of the ACM

    (1995)
  • F.G Halasz

    Reflections on notecards: seven issues for the next generation of hypermedia systems

    Communications of the ACM

    (1988)
  • E Hoffman et al.

    Experimental law and economics: an introduction

    Columbia Law Review

    (1985)
  • P.G Keen

    Computer-based decision aids: the evaluation problem

    Sloan Management Review

    (Spring 1975)
  • P.G Keen

    Adaptive design for decision support systems

    Database

    (1980)
  • P.G Keen

    Value analysis: justifying decision support systems

    MIS Quarterly

    (1981)
  • Cited by (0)

    Dr. James R. Marsden, the Shenkman Family Chair in e-Business, came to UConn in 1993 as Professor and Head, Department of Operations and Information Management, School of Business Administration, University of Connecticut. Dr. Marsden was part of a three-person concept development team that initiated and oversaw the development of the Connecticut Information Technology Institute and is currently serving as its Executive Director. He developed and implemented the Treibick Electronic Commerce Initiative that is funded through a generous gift provided by Richard Treibick and the Treibick Family Foundation. Dr. Marsden also serves as Director of the OPIM/SBA MIS Research Lab and is a member of Advisory Board and Steering Committee of CIBER (Center for International Business Education and Research). He was a member of the edgelab development team and currently serves on the edgelab Steering Committee which selects and resources projects and oversees operations. Dr. Marsden was a winner of the initial Chancellor's Award for IT Excellence and has a lengthy record in market innovation and analyses, economics of information, artificial intelligence, and production theory. His research work has appeared in Management Science; IEEE Transactions on Systems, Man, and Cybernetics; American Economic Review; Journal of Economic Theory; Journal of Political Economy; Computer Integrated Manufacturing Systems; Decision Support Systems; Journal of Management Information Systems, and numerous other academic journals. He was part of the IT Visioning and IT Planning Groups for the University and has played a leading role in developing the School of Business Administration as both a campus and national leader in IT education and research.

    Professor Marsden received his AB (Phi Beta Kappa, James Scholar, Evans Scholar) degree from the University of Illinois and his MSc and PhD degrees from Purdue University. Having completed his J.D. while at the University of Kentucky, Jim has been admitted to both the Kentucky and Connecticut Bar. He is an Area Editor of Decision Support Systems and serves in a frequent external evaluator for major U.S. and international universities. He has held visiting positions at the University of York (England), University of Arizona, Purdue University, and the University of North Carolina. Jim was an Invited Lecturer at two NATO Advanced Study Institutes on Decision Support Systems and has given keynote addresses and university seminars throughout Europe and the Far East.

    Ramakrishnan Pakath is an Associate Professor of Decision Science and Information Systems at the University of Kentucky. Ram holds an MSE (OR and IE) degree from The University of Texas at Austin and a PhD (Management-MIS) degree from Purdue University. His research focuses on (a) designing and evaluating adaptive problem processors, and (b) assessing information source impacts on system user performance. Dr. Pakath's research articles have appeared in such refereed forums as Decision Sciences, Decision Support Systems, European Journal of Operational Research, IEEE Transactions on Systems, Man, and Cybernetics, Information and Management, and Information Systems Research. He is author of the book Business Support Systems: An Introduction published by Copley, now in its second edition. Dr. Pakath has also contributed refereed material to a number of well-known books including Handbook of Industrial Engineering, Multimedia Technology and Applications, and Operations Research and Artificial Intelligence. He served as Director of the MIS Research Laboratory of the College of Business and Economics, University of Kentucky from 1993 to 1997. He is an Associate Editor for Decision Support Systems and an Editorial Board Member of Journal of End User Computing and Management.

    Kustim Wibowo is an Associate Professor in the MIS and Decision Sciences Department of the Eberly College of Business and IT, Indiana University of Pennsylvania. He received his PhD in MIS from the University of Kentucky. Dr. Wibowo also holds an MSc in Computer Science from Baylor University. His current research interests include: e-commerce and web security, information systems for educational technology, human resource information systems, and OLAP (OnLine Analytical Processing) for managerial decision support.

    1

    This author's work was supported by the Treibick Electronic Commerce Initiative at the University of Connecticut.

    2

    Their work was supported by a grant from the MIS Research Laboratory Endowment at the University of Kentucky.

    3

    Presently at MIS and Decision Sciences, Eberly College of Business, Indiana University of Pennsylvania, Indiana, PA 15701.

    View full text