skip to main content
10.1145/3025453.3025596acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

A Cognitive Model of How People Make Decisions Through Interaction with Visual Displays

Published: 02 May 2017 Publication History

Abstract

In this paper we report a cognitive model of how people make decisions through interaction. The model is based on the assumption that interaction for decision making is an example of a Partially Observable Markov Decision Process (POMDP) in which observations are made by limited perceptual systems that model human foveated vision and decisions are made by strategies that are adapted to the task. We illustrate the model by applying it to the task of determining whether to block a credit card given a number of variables including the location of a transaction, its amount, and the customer history. Each of these variables have a different validity and users may weight them accordingly. The model solves the POMDP by learning patterns of eye movements (strategies) adapted to different presentations of the data. We compare the model behavior to human performance on the credit card transaction task.

References

[1]
Chris Baber, Simon Attfield, Gareth Conway, Chris Rooney, and Neesha Kodagoda. 2016. Collaborative sense-making during simulated Intelligence Analysis exercises. International Journal of Human-Computer Studies 86 (2016), 94--108.
[2]
Robert W Baloh, Andrew W Sills, Warren E Kumley, and Vicente Honrubia. 1975. Quantitative measurement of saccade amplitude, duration, and velocity. Neurology 25, 11 (1975), 1065.
[3]
Arndt Bröder and Wolfgang Gaissmaier. 2011. Heuristics: The foundations of adaptive behavior. Oxford University Press.
[4]
Arndt Bröder and Stefanie Schiffer. 2006. Adaptive fiexibility and maladaptive routines in selecting fast and frugal decision strategies. Journal of Experimental Psychology: Learning, Memory, and Cognition 32, 4 (2006), 904.
[5]
Jerome R Busemeyer and James T Townsend. 1993. Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. Psychological review 100, 3 (1993), 432.
[6]
Nicholas J. Butko and Javier R. Movellan. 2008. I-POMDP: An infomax model of eye movement. In 2008 IEEE 7th International Conference on Development and Learning, ICDL. 139--144.
[7]
Xiuli Chen, Gilles Bailly, Duncan P. Brumby, Antti Oulasvirta, and Andrew Howes. 2015. The Emergence of Interactive Behavior: A Model of Rational Menu Search. Proceedings of the ACM CHI'15 Conference on Human Factors in Computing Systems 1 (2015), 4217--4226.
[8]
Andrea Dal Pozzolo, Olivier Caelen, Yann-Aël Le Borgne, Serge Waterschoot, and Gianluca Bontempi. 2014. Learned lessons in credit card fraud detection from a practitioner perspective. Expert systems with applications 41, 10 (2014), 4915--4928.
[9]
Thomas H. Davenport. 2013. How P&G Presents Data to Decision-Makers. Harvard Business Review (2013).
[10]
Fermin del Prado Martin. 2008. A theory of reaction time distributions. (dec 2008).
[11]
Paul Dourish. 2006. Implications for design. SIGCHI Conference on Human Factors in Computing Systems (CHI'06) (2006), 541--550.
[12]
John M. Findlay. 1982. Global visual processing for saccadic eye movements. Vision Research 22, 8 (1982), 1033--1045.
[13]
Wilson S Geisler. 2011. Contributions of ideal observer theory to vision research. Vision research 51, 7 (2011), 771--781.
[14]
Gerd Gigerenzer and Wolfgang Gaissmaier. 2011. Heuristic decision making. Annual review of psychology 62 (2011), 451--482.
[15]
Gerd Gigerenzer and Daniel G Goldstein. 1996. Reasoning the fast and frugal way: models of bounded rationality. Psychological review 103, 4 (1996), 650.
[16]
Gerd Gigerenzer and Peter M Todd. 1999. Fast and frugal heuristics: The adaptive toolbox. Oxford University Press.
[17]
Wayne D Gray, Chris R Sims, Wai-Tat W-T Fu, and Michael J Schoelles. 2006. The soft constraints hypothesis: a rational analysis approach to resource allocation for interactive behavior. Psych Review 113, 3 (2006), 461.
[18]
Mary Hayhoe and Dana Ballard. 2014. Modeling Task Control of Eye Movements. Current Biology 24, 13 (2014), R622--R628.
[19]
Andrew Howes, Geoffrey B Duggan, Kiran Kalidindi, Yuan-Chi Tseng, and Richard L Lewis. 2015. Predicting Short-Term Remembering as Boundedly Optimal Strategy Choice. Cognitive Science 40, 5 (2015), 1192--1223.
[20]
Andrew Howes, Richard L. Lewis, and Alonso Vera. 2009. Rational adaptation under task and processing constraints: implications for testing theories of cognition and action. Psychological review 116, 4 (2009), 717--751.
[21]
Sanjeev Jha and J Christopher Westland. 2013. A Descriptive Study of Credit Card Fraud Pattern. Global Business Review 14, 3 (2013), 373--384.
[22]
L Kaelbling, Michael L. Littman, and Ar Cassandra. 1998. Planning and Acting in Partially Observable Stochastic Domains. Artificial Intelligence 101, 1--2 (1998), 99--134.
[23]
David E. Kieras and Anthony J. Hornof. 2014. Towards accurate and practical predictive models of active-vision-based visual search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3875--3884.
[24]
Michael D Lee and S A Zhang. 2012. Evaluating the coherence of Take-the-best in structured environments. Judgment and Decision Making 7, 4 (2012).
[25]
Richard L. Lewis, Andrew Howes, and Satinder Singh. 2014. Computational rationality: linking mechanism and behavior through bounded utility maximization. Topics in Cognitive Science 6, 2 (2014), 279--311.
[26]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).
[27]
Ben R Newell and David R Shanks. 2003. Take the best or look at the rest? Factors influencing" one-reason" decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition 29, 1 (2003), 53.
[28]
Ben R Newell, Nicola J Weston, and David R Shanks. 2003. Empirical tests of a fast-and-frugal heuristic: Not everyone "takes-the-best". Organizational Behavior and Human Decision Processes 91, 1 (2003), 82--96.
[29]
Jose Nunez-Varela and Jeremy L. Wyatt. 2013. Models of gaze control for manipulation tasks. ACM Transactions on Applied Perception (TAP) 10, 4 (2013), 20.
[30]
Manoj Pandey. 2010. Operational risk forum: A model for managing online fraud risk using transaction validation. Journal of Operational Risk 1 (2010), 49.
[31]
Stephen J. Payne and Andrew Howes. 2013. Adaptive interaction: A utility maximization approach to understanding human interaction with technology. Synthesis Lectures on Human-Centered Informatics 6, 1 (2013), 1--111.
[32]
Rajesh P N Rao. 2010. Decision making under uncertainty: a neural model based on partially observable markov decision processes. Frontiers in computational neuroscience 4, November (2010), 146.
[33]
Roger Ratcliff. 2002. A diffusion model account of response time and accuracy in a brightness discrimination task: fitting real data and failing to fit fake but plausible data. Psychonomic bulletin {&} review 9, 2 (jun 2002), 278--291.
[34]
Roger Ratcliff and Philip L Smith. 2010. Perceptual discrimination in static and dynamic noise: the temporal relation between perceptual encoding and decision making. Journal of experimental psychology. General 139, 1 (feb 2010), 70--94.
[35]
Jörg Rieskamp and Ulrich Hoffrage. 2008. Inferences under time pressure: How opportunity costs affect strategy selection. Acta psychologica 127, 2 (2008), 258--276.
[36]
Jörg Rieskamp and Philipp E Otto. 2006. SSL: a theory of how people learn to select strategies. Journal of Experimental Psychology: General 135, 2 (2006), 207.
[37]
Daniel M Russell, Mark J Stefik, Peter Pirolli, and Stuart K Card. 1993. The cost structure of sensemaking. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems. ACM, 269--276.
[38]
Daniel Sánchez, M A Vila, L Cerda, and José-Maria Serrano. 2009. Association rules applied to credit card fraud detection. Expert Systems with Applications 36, 2 (2009), 3630--3640.
[39]
Guy Shani, Joelle Pineau, and Robert Kaplow. 2013. A survey of point-based POMDP solvers. Autonomous Agents and Multi-Agent Systems 27, 1 (2013), 1--51.
[40]
Nathan Sprague, Dana Ballard, and Al Robinson. 2007. Modeling embodied visual behaviors. ACM Transactions on Applied Perception (TAP) 4, 2 (2007), 11.
[41]
R.S. Sutton and A.G. Barto. 1998. Reinforcement Learning: An Introduction. IEEE Transactions on Neural Networks 9, 5 (1998), 1054--1054.
[42]
Julia Trommershäuser, Paul W Glimcher, and Karl R Gegenfurtner. 2009. Visual processing, learning and feedback in the primate eye movement system. Trends in Neurosciences 32, 11 (2009), 583--590.
[43]
Yuan-Chi Tseng and Andrew Howes. 2015. The adaptation of visual search to utility, ecology and design. International Journal of Human-Computer Studies 80 (2015), 45--55.
[44]
S Van der Stigchel and T C Nijboer. 2011. The global effect: what determines where the eyes land? Journal of Eye Movement Research 4, 2 (2011), 1--13.
[45]
Dustin Venini, Roger W. Remington, Gernot Horstmann, and Stefanie I. Becker. 2014. Centre-of-gravity fixations in visual search: When looking at nothing helps to find something. Journal of Ophthalmology 2014, June (2014).
[46]
Françoise Vitu. 2008. About the global effect and the critical role of retinal eccentricity: Implications for eye movements in reading. Journal of Eye Movement Research 2, 3 (2008), 1--18.
[47]
C Watkins and Peter Dayan. 1992. Q-Learning. Machine Learning 8 (1992), 279--292.

Cited By

View all
  • (2024)How can Artificial Intelligence Teammates Know What Humans Want? Using Eye-Tracking Data to Infer Human Preferences in Game-Theoretic Decision TasksProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/1071181324126683368:1(1542-1548)Online publication date: 24-Oct-2024
  • (2024)Naturalistic Digital Behavior Predicts Cognitive AbilitiesACM Transactions on Computer-Human Interaction10.1145/366034131:3(1-32)Online publication date: 7-May-2024
  • (2024)SIM2VR: Towards Automated Biomechanical Testing in VRProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676452(1-15)Online publication date: 13-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
May 2017
7138 pages
ISBN:9781450346559
DOI:10.1145/3025453
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2017

Permissions

Request permissions for this article.

Check for updates

Badges

  • Honorable Mention

Author Tags

  1. cognitive modeling
  2. decision making.
  3. eye movements
  4. markov decision process
  5. reinforcement learning
  6. visual search

Qualifiers

  • Research-article

Funding Sources

  • EU

Conference

CHI '17
Sponsor:

Acceptance Rates

CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;
Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)111
  • Downloads (Last 6 weeks)8
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)How can Artificial Intelligence Teammates Know What Humans Want? Using Eye-Tracking Data to Infer Human Preferences in Game-Theoretic Decision TasksProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/1071181324126683368:1(1542-1548)Online publication date: 24-Oct-2024
  • (2024)Naturalistic Digital Behavior Predicts Cognitive AbilitiesACM Transactions on Computer-Human Interaction10.1145/366034131:3(1-32)Online publication date: 7-May-2024
  • (2024)SIM2VR: Towards Automated Biomechanical Testing in VRProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676452(1-15)Online publication date: 13-Oct-2024
  • (2024)CRTypist: Simulating Touchscreen Typing Behavior via Computational RationalityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642918(1-17)Online publication date: 11-May-2024
  • (2024)Supporting Task Switching with Reinforcement LearningProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642063(1-18)Online publication date: 11-May-2024
  • (2024)Simulating Emotions With an Integrated Computational Model of Appraisal and Reinforcement LearningProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641908(1-12)Online publication date: 11-May-2024
  • (2024)Modeling the impact of mental models on interactive decision-making and behavior2024 16th International Conference on Human System Interaction (HSI)10.1109/HSI61632.2024.10613586(1-7)Online publication date: 8-Jul-2024
  • (2024)Understanding the Effects of Visual Impairment on Visual SearchUniversal Access in Human-Computer Interaction10.1007/978-3-031-60884-1_25(363-381)Online publication date: 1-Jun-2024
  • (2023)A Bayesian Approach for Quantifying Data Scarcity when Modeling Human Behavior via Inverse Reinforcement LearningACM Transactions on Computer-Human Interaction10.1145/355138830:1(1-27)Online publication date: 7-Mar-2023
  • (2023)Amortised Experimental Design and Parameter Estimation for User Models of PointingProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581483(1-17)Online publication date: 19-Apr-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media