Abstract
We present two visualization approaches illustrating the value of formal cognitive models for predicting, capturing, and understanding eye tracking as a manifestation of underlying cognitive processes and strategies. Computational cognitive models are formal theories of cognition which can provide predictions for human eye movements in visual decision-making tasks. Visualizing the internal dynamics of a model provides insights into how the interplay of cognitive mechanisms influences the observable eye movements. Animation of those model behaviors in virtual human agents gives explicit, high fidelity visualizations of model behavior, providing the analyst with an understanding of the simulated human’s behavior. Both can be compared to human data for insight about cognitive mechanisms engaged in visual tasks and how eye movements are affected by changes in internal cognitive strategies, external interface properties, and task demands. We illustrate the visualizations on two models of visual multitasking and juxtapose model performance against a human operator performing the same task.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The four tasks used herein constitute the standard implementation of the mMAT-B for multitasking research [6]. The number of tasks can be flexibly increased or decreased and the nature of the tasks can be changed to accommodate the research questions of interest. For more on the mMAT-B software, see http://sai.mindmodeling.org/mmatb.
References
Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychol. Rev. 111 (4), 1036–1060 (2004)
Blascheck, T., Kurzhals, K., Raschke, M., Burch, M., Weiskopf, D., Ertl. T.: State-of-the-art of visualization for eye tracking data. In: Proceedings of EuroVis, Swansea, vol. 2014 (2014)
Bojko, A.A.: Informative or misleading? Heatmaps deconstructed. In: Jacko, J.A. (eds.) Human-Computer Interaction. New Trends, pp. 30–39. Springer, Berlin/Heidelberg (2009)
Bostock, M., Ogievetsky, V., Heer, J.: D3: Data driven documents. IEEE Trans. Vis. Comput. Gr. 17 (12), 2301–2309 (2011)
Busemeyer, J.R., Diederich, A.: Cognitive Modeling. Sage, Los Angeles (2010)
Cline, J., Arendt, D.L., Geiselman, E.E., Blaha, L.M.: Web-based implementation of the modified multi-attribute task battery. In: 4th Annual Midwestern Cognitive Science Conference, Dayton (2014)
Courgeon, M., Rautureau, G., Martin, J.-C., Grynszpan, O.: Joint attention simulation using eye-tracking and virtual humans. IEEE Trans. Affect. Comput. 5 (3), 238–250 (2014)
Dam, E.B., Koch, M., Lillholm, M.: Quaternions, interpolation and animation. Technical report DIKU-TR-98/5, University of Copenhagen, Universitetsparken 1, DK-2100 Kbh (1998)
Halverson, T., Hornof, A.J.: A computational model of “Active Vision” for visual search in human-computer interaction. Hum. Comput. Interact. 26 (4), 285–314 (2011)
Halverson, T., Reynolds, B., Blaha, L.M.: SIMCog-JS: simplified interfacing for modeling cognition – JavaScript. In: Proceedings of the International Conference on Cognitive Modeling, Groningen, pp. 39–44 (2015)
Itti, L., Dhavale, N., Pighin, F.: Photorealistic attention-based gaze animation. In: 2006 IEEE International Conference on Multimedia and Expo, Toronto, pp. 521–524 (2006)
Kieras, D.E., Meyer, D.E.: An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Hum. Comput. Interact. 12 (4), 391–438 (1997)
Laird, J.E.: The Soar Cognitive Architecture. MIT Press, Cambridge (2012)
Nyamsuren, E., Taatgen, N.A.: Pre-attentive and Attentive Vision Module. In: Proceedings of the 2012 International Conference on Cognitive Modeling, Berlin, pp. 211–216 (2012)
Patterson, R.E., Blaha, L.M., Grinstein, G.G., Liggett, K.K., Kaveney, D.E., Sheldon, K.C., Havig, P.R., Moore, J.A.: A human cognition framework for information visualization. Comput. Gr. 42, 42–58 (2014)
Peebles, D.: A cognitive architecture-based model of graph comprehension. In: Rußwinkel, N., Drewitz, U., van Rijn, H. (eds.) 11th International Conference on Cognitive Modeling, Berlin, pp. 37–42 (2012)
Peebles, D.: Strategy and pattern recognition in expert comprehension of 2 × 2 interaction graphs. Cognit. Syst. Res. 24, 43–51 (2013)
Peebles, D., Cheng, P.C.-H.: Modeling the effect of task and graphical representation on response latency in a graph reading task. Human Factors: J. Hum. Fact. Ergon. Soc. 45 (1), 28–46 (2003)
Raschke, M., Blascheck, T., Richter, M., Agapkin, T., Ertl, T.: Visual analysis of perceptual and cognitive processes. In: 2014 International Conference on Information Visualization Theory and Applications (IVAPP), Lisbon, pp. 284–291 (2014)
Raschke, M., Engelhardt, S., Ertl, T.: A framework for simulating visual search strategies. In: Proceedings of the 11th International Conference on Cognitive Modeling, Ottawa, pp. 221–226 (2013)
Richardson, D.C., Dale, R.: Looking to understand: the coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cogn. Sci. 29 (6), 1045–1060 (2005)
Salvucci, D.D.: An integrated model of eye movements and visual encoding. Cogn. Syst. Res. 1 (4), 201–220 (2001)
Salvucci, D.D., Taatgen, N.A.: Threaded cognition: an integrated theory of concurrent multitasking. Psychol. Rev. 115 (1), 101–130 (2008)
Schoelles, M., Gray, W.D.: Speculations on model tracing for visual analytics. In: Proceedings of the 12th International Conference on Cognitive Modeling, Ottawa, pp. 406–407 (2013)
Teo, L.-H., John, B.E., Blackmon, M.H.: CogTool-Explorer: a model of goal-directed user exploration that considers information layout. In: Proceedings of the Conference on Human Factors in Computing Systems, Texas, pp. 2479–2488 (2012)
Wang, L.-C., Chen, C.: A combined optimization method for solving the inverse kinematics problems of mechanical manipulators. IEEE Trans. Robot. Autom. 7 (4), 489–499 (1991)
Wolfe, J.M.: Guided Search 4.0. In: Gray, W.D. (ed.) Integrated Models of Cognitive Systems, pp. 99–119. Oxford University Press, Oxford/New York(2007)
Zhang, Y., Hornof, A.J.: Understanding multitasking through parallelized strategy exploration and individualized cognitive modeling. In: Conference on Human Factors in Computing Systems, New York, pp. 3885–3894 (2014)
Acknowledgements
The authors thank Dr. Megan Morris and Mr. Jacob Kern for their assistance in processing the Tobii Glasses data, and Dr. Dustin Arendt for recurrence plot inspiration. The views expressed in this paper are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government. This work was supported by AFOSR LRIR to L.M.B. Distribution A: Approved for public release; distribution unlimited. 88ABW Cleared 08/26/2015; 88ABW-2015-4021.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Balint, J.T., Reynolds, B., Blaha, L.M., Halverson, T. (2017). Visualizing Eye Movements in Formal Cognitive Models. In: Burch, M., Chuang, L., Fisher, B., Schmidt, A., Weiskopf, D. (eds) Eye Tracking and Visualization. ETVIS 2015. Mathematics and Visualization. Springer, Cham. https://doi.org/10.1007/978-3-319-47024-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-47024-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47023-8
Online ISBN: 978-3-319-47024-5
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)