Abstract
Research on mental models has been ongoing in Human-Computer Interaction (HCI) since the 1980s. A mental model derives from humans’ perception, memories, knowledge, and causal beliefs about the external reality, mediating how they describe, explain, and predict the world surrounding them. By understanding users’ mental models of a target system, designers can improve the learnability and usability of a system via a better interface. Although understanding mental models is important for interface design, unfortunately understanding what is meant by the term ‘mental model’ can be a challenge. The field has historically been plagued by inconsistent usage of the term, as pointed out by prior researchers. It has been some time since the last comprehensive evaluation of HCI-related mental model research. This scoping review provides a comprehensive overview of the current literature on mental model research in HCI from 2010 to 2021, exploring aspects such as where research has been conducted, by whom, on what, using which methods, and for which purposes. The findings suggest a growing diversity in approach, with new perspectives and methodologies being introduced. Our review was initiated by a desire to understand the state of the art of mental models research, and sustained by a sense of confusion about the different ways that different researchers used the term. We hope that it will help people new to the field to make sense of the rich, diverse, but at times rather confusing ways that the term ‘mental model’ is used in HCI research.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Holtrop, J.S., Scherer, L.D., Matlock, D.D., Glasgow, R.E., Green, L.A.: The importance of mental models in implementation science. Front. Publ. Health 9 (2021). https://www.frontiersin.org/article/10.3389/fpubh.2021.680316. Accessed 16 May 2022
Johnson-Laird, P.N.: The history of mental models. In: Psychology of Reasoning. Psychology Press (2004)
Lee, Y.-C., Malcein, L.A.: Users’ mental models for computer-mediated communication: theorizing emerging technology and behavior in eHealth applications. Hum. Behav. Emerg. Technol. 2(4), Article no. 4 (2020). https://doi.org/10.1002/hbe2.212
Rouse, W.B., Morris, N.M.: On looking into the black box: prospects and limits in the search for mental models. Psychol. Bull. 100(3), 349–363 (1986). https://doi.org/10.1037/0033-2909.100.3.349
Scheutz, M.: Computational mechanisms for mental models in human-robot interaction. In: Shumaker, R. (ed.) VAMR 2013. LNCS, vol. 8021, pp. 304–312. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39405-8_34
Wilson, J.R., Rutherford, A.: Mental models: theory and application in human factors. Hum. Factors 31(6), 617–634 (1989). https://doi.org/10.1177/001872088903100601
Norman, D.A.: Some observations on mental models. In: Mental Models. Psychology Press (1983)
Payne, S.J.: Users’ Mental Models: The Very Ideas. Elsevier Inc., Amsterdam (2003). https://doi.org/10.1016/B978-155860808-5/50006-X
Craik, K.J.W.: The Nature of Explanation, pp. viii, 123. University Press, Macmillan, Oxford (1943)
Gentner, D., Stevens, A.L.: Mental Models. Psychology Press (1983)
Johnson-Laird, P.N.: Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Harvard University Press (1983)
Card, S.K., Moran, T.P., Newell, A.: The Psychology of Human-Computer Interaction | Guide Books (1983). https://dl.acm.org/doi/abs/10.5555/578027. Accessed 13 Apr 2022
Ahmed, F.: Profile of Philip N. Johnson-Laird. Proc. . Acad. Sci. USA 108(50), 19862–19864 (2011). https://doi.org/10.1073/pnas.1117174108
Combefis, S., Giannakopoulou, D., Pecheur, C., Feary, M.: A formal framework for design and analysis of human-machine interaction. In: 2011 IEEE International Conference on 2011 IEEE International Conference on Systems, Man, and Cybernetics, Systems, Man, and Cybernetics (SMC), pp. 1801–1808, October 2011. https://doi.org/10.1109/ICSMC.2011.6083933
Rowe, A.L., Cooke, N.J., Neville, K.J., Schacherer, C.W.: Mental models of metal models: a comparison of mental model measurement techniques. In: Proceedings of the Human Factors Society Annual Meeting, vol. 36, no. 16, pp. 1195–1199, October 1992. https://doi.org/10.1177/154193129203601603
Cooke, N.J., Rowe, A.L.: Evaluating mental model elicitation methods. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 38, no. 4, Art. no. 4, October 1994. https://doi.org/10.1177/154193129403800416
Paivio, A.: Mental Representations: A Dual Coding Approach. Oxford University Press, Oxford (1990)
Sasse, M.A.: Eliciting and describing users’ models of computer systems. Electronic thesis or dissertation (1997). [Online]
Nielsen, J.: A meta-model for interacting with computers. Interact. Comput.Comput. 2(2), 147–160 (1990). https://doi.org/10.1016/0953-5438(90)90020-I
Arksey, H., O’Malley, L.: Scoping Studies: Towards a Methodological Framework. vol. 8, no. 1, pp. 19–32, February 2005. https://doi.org/10.1080/1364557032000119616
Pham, M.T., Rajić, A., Greig, J.D., Sargeant, J.M., Papadopoulos, A., McEwen, S.A.: A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res. Synth. Methods 5(4), 371–385 (2014). https://doi.org/10.1002/jrsm.1123
Long, R., Zhang, J.: Research on information architecture based on graphic reasoning and mental model. In: Stephanidis, C. (ed.) HCI 2018. CCIS, vol. 850, pp. 85–92. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92270-6_12
Khairuddin, I.E., Sas, C., Speed, C.: BlocKit: a physical kit for materializing and designing for blockchain infrastructure. In: DIS – Proceedings of ACM Designing Interactive Systems Conference, pp. 1449–1462 (2019). https://doi.org/10.1145/3322276.3322370
Long, R., Liu, X., Lei, T., Chen, X., Jin, Z.: The impact of chinese traditional cultural on the gesture and user experience in mobile interaction design. In: Rau, P.-L.P. (ed.) CCD 2017. LNCS, vol. 10281, pp. 49–58. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57931-3_5
Renzi, A.B., Freitas, S.: Affordances and gestural interaction on multi-touch interface systems: building new mental models. In: Marcus, A. (ed.) DUXU 2014. LNCS, vol. 8518, pp. 615–623. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07626-3_58
Du, Y., Qin, J., Zhang, S., Cao, S., Dou, J.: Voice user interface interaction design research based on user mental model in autonomous vehicle. In: Kurosu, M. (ed.) HCI 2018. LNCS, vol. 10903, pp. 117–132. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91250-9_10
Dechand, S., Naiakshina, A., Danilova, A., Smith, M.: In encryption we don’t trust: the effect of end-to-end encryption to the masses on user perception. In: Proceedings - IEEE European Symposium on Security and Privacy, EURO S P, pp. 401–415 (2019). https://doi.org/10.1109/EuroSP.2019.00037
Prettyman, S.S., Furman, S., Theofanos, M., Stanton, B.: Privacy and security in the brave new world: the use of multiple mental models. In: Tryfonas, T., Askoxylakis, I. (eds.) HAS 2015. LNCS, vol. 9190, pp. 260–270. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20376-8_24
Bertrand, P., Guegan, J., Robieux, L., McCall, C.A., Zenasni, F.: Learning empathy through virtual reality: multiple strategies for training empathy-related abilities using body ownership illusions in embodied virtual reality. Front. Robot. AI 5 (2018). https://doi.org/10.3389/frobt.2018.00026
Vlist, B., Niezen, G., Rapp, S., Hu, J., Feijs, L.: Configuring and controlling ubiquitous computing infrastructure with semantic connections: a tangible and an AR approach. Pers. Ubiquit. Comput.Ubiquit. Comput. 17(4), 783–799 (2013)
Van Kerckhoven, J., Geldof, S., Vermeersch, B.: Contextual inquiry in signal boxes of a railway organization. In: Palanque, P., Vanderdonckt, J., Winckler, M. (eds.) HESSD 2009. LNCS, vol. 5962, pp. 96–106. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-11750-3_8
Rutjes, H., Willemsen, M., IJsselsteijn, W.: “Considerations on explainable AI and users’ mental models. In: Where is the Human? Bridging the Gap Between AI and HCI: Workshop at CHI’19, 4–9 May 2019, Glasgow, Scotland UK, May 2019. https://research.tue.nl/en/publications/considerations-on-explainable-ai-and-users-mental-models. Accessed 10 Jan 2022
Coopamootoo, K.P.L., Groß, T.: Mental models for usable privacy: a position paper. In: Tryfonas, T., Askoxylakis, I. (eds.) HAS 2014. LNCS, vol. 8533, pp. 410–421. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07620-1_36
Johns, J.D.: Strategic design: breaking mental models initiates learning in video games. In: Zaphiris, P., Ioannou, A. (eds.) LCT 2017. LNCS, vol. 10295, pp. 443–461. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58509-3_35
Chunpir, H.I., Ludwig, T.: A software to capture mental models. In: Antona, M., Stephanidis, C. (eds.) UAHCI 2017. LNCS, vol. 10279, pp. 393–409. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58700-4_32
Pantförder, D., Schaupp, J., Vogel-Heuser, B.: Making implicit knowledge explicit – acquisition of plant staff’s mental models as a basis for developing a decision support system. In: Stephanidis, C. (ed.) HCI 2017. CCIS, vol. 713, pp. 358–365. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58750-9_50
Lei, T., Liu, X., Wu, L., Jin, Z., Wang, Y., Wei, S.: The influence of matching degree of the user’s inherent mental model and the product’s embedded mental model on the mobile user experience. In: Kurosu, M. (ed.) HCI 2016. LNCS, vol. 9732, pp. 320–329. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39516-6_31
Nishimoto, H., Koyanagi, T., Sarata, M., Kinoshita, A., Okuda, M.: “Memes” UX-design methodology based on cognitive science regarding instrumental activities of daily living. In: Duffy, V.G. (ed.) HCII 2019. LNCS, vol. 11582, pp. 264–273. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22219-2_20
Phillips, R., Lockton, D., Baurley, S., Silve, S.: Making instructions for others: exploring mental models through a simple exercise. In: Interactions 20(5), Article no. 5 (2013). https://doi.org/10.1145/2505290
Chammas, A., Quaresma, M., Mont’Alvão, C.R.: Children’s mental model as a tool to provide innovation in digital products. In: Marcus, A. (ed.) DUXU 2015. LNCS, vol. 9187, pp. 23–33. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20898-5_3
Hvorecký, J., Drlík, M., Munk, M.: Enhancing database querying skills by choosing a more appropriate interface. In: IEEE EDUCON 2010 Conference, Education Engineering (EDUCON), April 2010, pp. 1897–1905. IEEE (2010). https://doi.org/10.1109/EDUCON.2010.5492434
Varga, E., Pattynama, P., Freudenthal, A.: Manipulation of mental models of anatomy in interventional radiology and its consequences for design of human-computer interaction. Cogn. Technol. Work 15(4), Article no. 4 (2013)
Yarosh, S., Zave, P.: Locked or not? Mental models of IoT feature interaction. In: Conference on Human Factors in Computing Systems and Processing, vol. 2017-May, pp. 2993–2997 (2017). https://doi.org/10.1145/3025453.3025617
van den Bosch, K., Schoonderwoerd, T., Blankendaal, R., Neerincx, M.: Six challenges for human-AI co-learning. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2019. LNCS, vol. 11597, pp. 572–589. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22341-0_45
Beimel, D., Kedmi-Shahar, E.: Improving the identification of functional system requirements when novice analysts create use case diagrams: the benefits of applying conceptual mental models. Requir. Eng. 24(4), Article no. 4 (2019). https://doi.org/10.1007/s00766-018-0296-z
Dou, J., Qin, J.: Research on user mental model acquisition based on multidimensional data collaborative analysis in product service system innovation process. In: Harris, D. (ed.) EPCE 2017. LNCS (LNAI), vol. 10276, pp. 35–44. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58475-1_3
Loeffler, D., Hurtienne, J., Hess, A., Maier, A., Schmitt, H.: Developing intuitive user interfaces by integrating users’ mental models into requirements engineering. In: HCI 2013 - 27th International British Computer Society Human Computer Interaction Conference: The Internet of Things (2013). https://doi.org/10.14236/ewic/hci2013.14
Qian, X., Yang, Y., Gong, Y.: The art of metaphor: a method for interface design based on mental models. In: Proceedings of VRCAI: ACM SIGGRAPH Conference on Virtual Reality and Continuum Its Applications Industry, Hong Kong, pp. 171–178 (2011). https://doi.org/10.1145/2087756.2087780
Revell, K.M.A., Stanton, N.A.: Models of models: filtering and bias rings in depiction of knowledge structures and their implications for design. Ergonomics 55(9), Article no. 9, Sep. 2012
Coopamootoo, K.P.L., Groß, T.: Mental models of online privacy: structural properties with cognitive maps. In: Proceedings of the 28th International BCS Human Computer Interaction Conference: Sand, Sea and Sky - Holiday HCI, HCI 2014, pp. 287–292 (2014). https://doi.org/10.14236/ewic/hci2014.46
Gondomar, R., Mor, E.: Understanding agency in human-computer interaction design. In: Kurosu, M. (ed.) HCII 2021. LNCS, vol. 12762, pp. 137–149. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78462-1_10
Payne, S.J.: Mental models in human-computer interaction. In: The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications, 3rd edn, pp. 41–54. CRC Press, Boca Raton (2012). https://doi.org/10.1201/b11963-ch-3
Heuer, T., Stein, J.: From HCI to HRI: about users, acceptance and emotions. In: Ahram, T., Karwowski, W., Pickl, S., Taiar, R. (eds.) IHSED 2019. AISC, vol. 1026, pp. 149–153. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-27928-8_23
Albalawi, T.F.: Quantifying the effect of cognitive biases on security decision-making (2018)
Kulesza, T., Burnett, M., Kwan, I., Stumpf, S.: Tell me more? The effects of mental model soundness on personalizing an intelligent agent. I:n Conference on Human Factors in Computing Systems - Proceedings, January 2012, pp. 1–10 (2012). https://doi.org/10.1145/2207676.2207678
Doi, T., Ishihara, K., Yamaoka, T.: Verification of the questionnaire for the level of mental models building. In: Stephanidis, C. (ed.) HCI 2013. CCIS, vol. 373, pp. 105–108. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39473-7_22
Jones, N., Ross, H., Lynam, T., Perez, P., Leitch, A.: Mental models: an interdisciplinary synthesis of theory and methods. Ecol. Soc. 16(1), Article no. 1 (2011). https://doi.org/10.5751/ES-03802-160146
Zhang, L., Zhang, X., Duan, Y., Fu, Z., Wang, Y.: Evaluation of learning performance of e-learning in China: a methodology based on change of internal mental model of learners. Turk. Online J. Educ. Technol. - TOJET 9(1), 70–82 (2010)
Spero, E., Stojmenović, M., Biddle, R.: Helping users secure their data by supporting mental models of VeraCrypt. In: Stephanidis, C. (ed.) HCII 2019. CCIS, vol. 1032, pp. 211–218. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23522-2_27
Valdez, A.C., Ziefle, M., Sedlmair, M.: Priming and anchoring effects in visualization. IEEE Trans. Vis. Comput. Graph.Comput. Graph. 24(1), 584–594 (2018). https://doi.org/10.1109/TVCG.2017.2744138
Lei, T., et al.: Gestures: the reformer of the user’s mental model in mobile HCI. In: Design, User Experience & Usability. User Experience Design for Diverse Interaction Platforms & Environments, pp. 586–597, January 2014
Nelles, J., Kuz, S., Schlick, C.M.: Ergonomic visualization of logistical control parameters for flexible production planning and control in future manufacturing systems. In: Stephanidis, C. (ed.) HCI 2015. CCIS, vol. 529, pp. 684–689. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21383-5_116
Filippi, S., Barattin, D.: Considering users’ different knowledge about products to improve a UX evaluation method based on mental models. In: Marcus, A., Wang, W. (eds.) DUXU 2018. LNCS, vol. 10918, pp. 367–378. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91797-9_26
Jenness, J., Lenneman, J., Benedick, A., Huey, R., Jaffe, J., Singer, J., Yahoodik, S.: Spaceship, guardian, coach: drivers’ mental models of advanced vehicle technology. In: Stephanidis, C. (ed.) HCII 2019. CCIS, vol. 1034, pp. 351–356. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23525-3_46
Di Mascio, T., Gennari, R., Tarantino, L., Vittorini, P.: Designing visualizations of temporal relations for children: action research meets HCI. Multimedia Tools Appl. 76(4), Article no. 4 (2017). https://doi.org/10.1007/s11042-016-3609-6
Xie, B., Zhou, J.: The influence of mental model similarity on user performance: comparing older and younger adults. In: Zhou, J., Salvendy, G. (eds.) ITAP 2017. LNCS, vol. 10298, pp. 569–579. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58536-9_45
Acemyan, C.Z., Kortum, P., Byrne, M.D., Wallach, D.S.: Users’ mental models for three end-to-end voting systems: Helios, Prêt à voter, and scantegrity II. In: Tryfonas, T., Askoxylakis, I. (eds.) HAS 2015. LNCS, vol. 9190, pp. 463–474. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20376-8_41
Zheng, Y., Zhao, G.: Natural interaction in video image investigation and its evaluation. In: Harris, D. (ed.) EPCE 2018. LNCS (LNAI), vol. 10906, pp. 520–532. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91122-9_42
Mlilo, S., Thatcher, A.: Mental models: have users’ mental models of web search engines improved in the last ten years? In: Harris, D. (ed.) EPCE 2011. LNCS (LNAI), vol. 6781, pp. 243–253. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21741-8_27
Nourani, M., et al.: Anchoring bias affects mental model formation and user reliance in explainable AI systems. In: International Conference on Intelligent User Interfaces Proceedings, IUI, pp. 340–350 (2021). https://doi.org/10.1145/3397481.3450639
NazariShirehjini, A.A., SoltaniNejad, F., Saniee-Monfared, G., Semsar, A., Shirmohammadi, S.: Mental model development using collaborative 3D virtual environments. In: Streitz, N., Markopoulos, P. (eds.) DAPI 2016. LNCS, vol. 9749, pp. 279–290. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39862-4_26
Lai, L.-L.: How do they tag? Senior adults’ tagging behavior in cultural heritage information. In: Nah, F.F.-H., Xiao, B.S. (eds.) HCIBGO 2018. LNCS, vol. 10923, pp. 475–484. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91716-0_38
Chen, Z., Su, Z.: Using participatory design methods to study users’ emotional experiences. In: Proceedings of International Conference on E-Business E-Goverence, ICEE, Guangzhou, pp. 425–428 (2010). https://doi.org/10.1109/ICEE.2010.115
González, V.M., Juárez, R.: How do you understand Twitter?: Analyzing mental models, understanding and learning about complex interactive systems. In: Collazos, C., Liborio, A., Rusu, C. (eds.) CLIHC 2013. LNCS, vol. 8278, pp. 103–110. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03068-5_18
Wohler, M., Loy, F., Schulte, A.: Mental models as common ground for human-agent interaction in cognitive assistant systems. In: HCI-Aero 2014 - Proceedings of the International Conference on Human-Computer Interaction in Aerospace, vol. 30 (2014). https://doi.org/10.1145/2669592.2669686
Goodrich, M.A., Yi, D.: Toward task-based mental models of human-robot teaming: a Bayesian approach. In: Shumaker, R. (ed.) VAMR 2013. LNCS, vol. 8021, pp. 267–279. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39405-8_30. http://www.library.illinois.edu/proxy/go.php?url=https://search.ebscohost.com/login.aspx?direct=true&db=edselc&AN=edselc.2-52.0-84884829119&site=eds-live&scope=site
Johnson-Laird, P.N.: Mental models and human reasoning. PNAS 107(43), Article no. 43 (2010). https://doi.org/10.1073/pnas.1012933107
Kramer, J., Noronha, S.: Designing with the user in mind a cognitive category based design methodology. In: Marcus, A. (ed.) DUXU 2014. LNCS, vol. 8520, pp. 152–163. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07638-6_16
Calero Valdez, A., Ziefle, M., Alagöz, F., Holzinger, A.: Mental models of menu structures in diabetes assistants. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds.) ICCHP 2010. LNCS, vol. 6180, pp. 584–591. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14100-3_87
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hu, X., Twidale, M. (2023). A Scoping Review of Mental Model Research in HCI from 2010 to 2021. In: Kurosu, M., et al. HCI International 2023 – Late Breaking Papers. HCII 2023. Lecture Notes in Computer Science, vol 14054. Springer, Cham. https://doi.org/10.1007/978-3-031-48038-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-48038-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48037-9
Online ISBN: 978-3-031-48038-6
eBook Packages: Computer ScienceComputer Science (R0)