Ja Eun Yu
Debaleena Chattopadhyay
ABSTRACT
Older adults often struggle to locate a function quickly in feature-rich user interfaces (UIs). Mobile UIs not only pack a ton of features in a small screen but also get frequent updates to their visual layouts—thereby exacerbating the problem. This paper explores a design solution where users could search for a UI feature using spoken-word queries. We investigated: 1) what type of questions older users ask when facing interaction challenges in unfamiliar scenarios, 2) how those query types compare with younger users’ inquiries, and 3) how older adults use a voice assistant design probe in a Wizard-of-Oz (WoZ) study. Results reveal five query types when verbally articulating interaction issues: validation, directed and undirected informational, navigational, and conceptual. In the WoZ study, older users typically asked for help following a series of non-unique or off-task feature selections (n = 13/15), and in 77% of those instances, they completed the task in the next interaction.
Supplemental Material
- AARP. 2021. AARP Foundation - for a future without senior poverty. Retrieved December 12, 2022 from https://www.aarp.org/aarp-foundation/.Google Scholar
- Sofianiza Abd Malik and Alistair DN Edwards. 2010. Investigation of cultural dependency in mobile technology and older adults. In CHI’10 Extended Abstracts on Human Factors in Computing Systems. 3835–3840.Google Scholar
- Akiko Aizawa. 2003. An information-theoretic perspective of tf–idf measures. Information Processing & Management 39, 1 (2003), 45–65.Google ScholarDigital Library
- Amos Azaria, Jayant Krishnamurthy, and Tom M Mitchell. 2016. Instructable Intelligent Personal Agent.. In AAAI, Vol. 4.Google Scholar
- Yvonne Barnard, Mike D Bradley, Frances Hodgson, and Ashley D Lloyd. 2013. Learning to use new technologies by older adults: Perceived difficulties, experimentation behaviour and usability. Computers in Human Behavior 29, 4 (2013), 1715–1724.Google ScholarDigital Library
- Eva Brandt, Thomas Binder, Lone Malmborg, and Tomas Sokoler. 2010. Communities of everyday practice and situated elderliness as an approach to co-design for senior interaction. In Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction. 400–403.Google ScholarDigital Library
- Virginia Braun and Victoria Clarke. 2012. Thematic analysis.(2012).Google Scholar
- Andrei Broder. 2002. A taxonomy of web search. In ACM Sigir forum, Vol. 36. ACM New York, NY, USA, 3–10.Google Scholar
- Fan Bu, Xingwei Zhu, Yu Hao, and Xiaoyan Zhu. 2010. Function-based question classification for general QA. In Proceedings of the 2010 conference on empirical methods in natural language processing. 1119–1128.Google Scholar
- L Buis, D Singer, E Solway, M Kirch, J Kullgren, and P Malani. 2020. Telehealth use among older adults before and during COVID-19. University of Michigan National Poll on Healthy Aging (2020).Google Scholar
- Debaleena Chattopadhyay, Francesca Salvadori, Kenton O’hara, and Sean Rintel. 2018. Beyond presentation: Shared slideware control as a resource for collocated collaboration. Human–Computer Interaction 33, 5-6 (2018), 455–498.Google Scholar
- Pei-Yu Chi, Sally Ahn, Amanda Ren, Mira Dontcheva, Wilmot Li, and Björn Hartmann. 2012. MixT: automatic generation of step-by-step mixed media tutorials. In Proceedings of the 25th annual ACM symposium on User interface software and technology. 93–102.Google ScholarDigital Library
- Sara J. Czaja, Walter R. Boot, Neil Charness, and Wendy A. Rogers. 2019. Designing for Older Adults: Principles and Creative Human Factors Approaches (3 ed.). CRC Press. https://doi.org/10.1201/b22189Google ScholarCross Ref
- Sara J Czaja and Chin Chin Lee. 2007. The impact of aging on access to technology. Universal access in the information society 5, 4 (2007), 341–349.Google ScholarDigital Library
- Mark Dunlop, Andreas Komninos, Emma Nicol, and Iain Hamiliton. 2014. Shake’n’Tap: a gesture enhanced keyboard for older adults. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services. 525–530.Google Scholar
- Rahhal Errattahi, Asmaa El Hannani, and Hassan Ouahmane. 2018. Automatic speech recognition errors detection and correction: A review. Procedia Computer Science 128 (2018), 32–37.Google ScholarCross Ref
- Mingming Fan, Qiwen Zhao, and Vinita Tibdewal. 2021. Older adults’ think-aloud verbalizations and speech features for identifying user experience problems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarDigital Library
- Carolyn L Foss. 1989. Detecting lost users: Empirical studies on browsing hypertext.Google Scholar
- Helene Gelderblom, Tobie van Dyk, and Judy van Biljon. 2010. Mobile phone adoption: Do existing models adequately capture the actual usage of older adults?. In Proceedings of the 2010 annual research conference of the south african institute of computer scientists and information technologists. 67–74.Google ScholarDigital Library
- Ido Guy. 2016. Searching by talking: Analysis of voice queries on mobile web search. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 35–44.Google ScholarDigital Library
- Graeme S Halford, Nelson Cowan, and Glenda Andrews. 2007. Separating cognitive capacity from knowledge: A new hypothesis. Trends in cognitive sciences 11, 6 (2007), 236–242.Google Scholar
- Christina N Harrington, Radhika Garg, Amanda Woodward, and Dimitri Williams. 2022. “It’s Kind of Like Code-Switching”: Black Older Adults’ Experiences with a Voice Assistant for Health Information Seeking. (2022).Google Scholar
- Rebecca Hill, Paul Beynon-Davies, and Michael D Williams. 2008. Older people and internet engagement: Acknowledging social moderators of internet adoption, access and use. Information Technology & People(2008).Google Scholar
- Bora Jin, Junghwan Kim, and Lisa M Baumgartner. 2019. Informal learning of older adults in using mobile devices: A review of the literature. Adult Education Quarterly 69, 2 (2019), 120–141.Google ScholarCross Ref
- Xiaofu Jin, Xiaozhu Hu, Xiaoying Wei, and Mingming Fan. 2022. Synapse: Interactive Guidance by Demonstration with Trial-and-Error Support for Older Adults to Use Smartphone Apps. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1–24.Google ScholarDigital Library
- Brittne Kakulla. 2021. 2020 Tech Ownership and Use. Retrieved December 12, 2022 from https://doi.org/10.26419/res.00420.004.Google ScholarCross Ref
- Brittne Kakulla. 2021. 2021 Barriers and Digital Inequities. Retrieved December 12, 2022 from https://doi.org/10.26419/res.00420.003.Google ScholarCross Ref
- Brittne Kakulla. 2021. 2021 Tech Trends and the 50-Plus: Top 10 Biggest Trends.Retrieved December 12, 2022 from https://doi.org/10.26419/res.00420.011.Google ScholarCross Ref
- B Kakulla. 2021. 2021 Tech Trends and the 50+: Top 10 Biggest Trends.Google Scholar
- Brittne Kakulla. 2021. 2022 Tech Trends and Adults 50-Plus.Retrieved December 12, 2022 from https://doi.org/10.26419/res.00493.001.Google ScholarCross Ref
- Pegah Karimi, Kallista Ballard, Pooja Vazirani, Ravi Teja Narasimha Jorigay, and Aqueasha Martin-Hammond. 2022. Designing Conversational Assistants to Support Older Adults’ Personal Health Record Access. In International Conference on Pervasive Computing Technologies for Healthcare. Springer, 253–271.Google Scholar
- Sunyoung Kim and Abhishek Choudhury. 2021. Exploring older adults’ perception and use of smart speaker-based voice assistants: A longitudinal study. Computers in Human Behavior 124 (2021), 106914.Google ScholarDigital Library
- Bran Knowles and Vicki L Hanson. 2018. The Wisdom of Older Technology (Non-) Users. Commun. ACM 61, 3 (2018), 72–77.Google ScholarDigital Library
- Bran Knowles, Vicki L Hanson, Yvonne Rogers, Anne Marie Piper, Jenny Waycott, Nigel Davies, Aloha Hufana Ambe, Robin N Brewer, Debaleena Chattopadhyay, Marianne Dee, 2021. The harm in conflating aging with accessibility. Commun. ACM 64, 7 (2021), 66–71.Google ScholarDigital Library
- Mandy Korzetz, Romina Kühn, Lukas Büschel, Franz-Wilhelm Schumann, Uwe Aßmann, and Thomas Schlegel. 2020. Introducing Mobile Device-Based Interactions to Users: An Investigation of Onboarding Tutorials. In International Conference on Human-Computer Interaction. Springer, 428–442.Google ScholarDigital Library
- Qingchuan Li and Yan Luximon. 2016. Older adults and digital technology: A study of user perception and usage behavior. In Advances in physical ergonomics and human factors. Springer, 155–163.Google Scholar
- Qingchuan Li and Yan Luximon. 2020. Older adults’ use of mobile device: usability challenges while navigating various interfaces. Behaviour & Information Technology 39, 8 (2020), 837–861.Google ScholarCross Ref
- Qingchuan Li, Yan Luximon, 2018. Understanding older adults’ post-adoption usage behavior and perceptions of mobile technology. (2018).Google Scholar
- Toby Jia-Jun Li, Amos Azaria, and Brad A Myers. 2017. SUGILITE: creating multimodal smartphone automation by demonstration. In Proceedings of the 2017 CHI conference on human factors in computing systems. 6038–6049.Google Scholar
- Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, and Jason Baldridge. 2020. Mapping natural language instructions to mobile UI action sequences. arXiv preprint arXiv:2005.03776(2020).Google Scholar
- Jiao Liu, Yanling Li, and Min Lin. 2019. Review of intent detection methods in the human-machine dialogue system. In Journal of Physics: Conference Series, Vol. 1267. IOP Publishing, 012059.Google Scholar
- Shareen Mahmud, Jessalyn Alvina, Parmit K Chilana, Andrea Bunt, and Joanna McGrenere. 2020. Learning through exploration: how children, adults, and older adults interact with a new feature-rich application. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.Google ScholarDigital Library
- Matej Martinc, Blaž Škrlj, and Senja Pollak. 2020. TNT-KID: Transformer-based neural tagger for keyword identification. Natural Language Engineering(2020), 1–40.Google Scholar
- Rui Mata, Lael J Schooler, and Jörg Rieskamp. 2007. The aging decision maker: cognitive aging and the adaptive selection of decision strategies.Psychology and aging 22, 4 (2007), 796.Google Scholar
- Carolyn Pang, Zhiqin Collin Wang, Joanna McGrenere, Rock Leung, Jiamin Dai, and Karyn Moffatt. 2021. Technology Adoption and Learning Preferences for Older Adults: Evolving Perceptions, Ongoing Challenges, and Emerging Design Opportunities. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarDigital Library
- Andraž Petrovčič, Walter R Boot, Tomaž Burnik, and Vesna Vesna Dolničar. 2019. Improving the Measurement of Older Adults’ Mobile Device Proficiency: Results and Implications from a Study of Older Adult Smartphone Users. IEEE Access 7(2019), 150412–150422.Google ScholarCross Ref
- Research Center Pew. 2021. Mobile Fact Sheet. Retrieved October, 2021 from https://www.pewresearch.org/internet/fact-sheet/mobile/.Google Scholar
- Anne Marie Piper, Raymundo Cornejo Garcia, and Robin N Brewer. 2016. Understanding the challenges and opportunities of smart mobile devices among the oldest old. International Journal of Mobile Human Computer Interaction (IJMHCI) 8, 2(2016), 83–98.Google ScholarDigital Library
- Alisha Pradhan, Amanda Lazar, and Leah Findlater. 2020. Use of intelligent voice assistants by older adults with low technology use. ACM Transactions on Computer-Human Interaction (TOCHI) 27, 4(2020), 1–27.Google ScholarDigital Library
- Jennifer C Romano Bergstrom, Erica L Olmsted-Hawala, and Matt E Jans. 2013. Age-related differences in eye tracking and usability performance: website usability for older adults. International Journal of Human-Computer Interaction 29, 8(2013), 541–548.Google ScholarCross Ref
- Daniel E Rose and Danny Levinson. 2004. Understanding user goals in web search. In Proceedings of the 13th international conference on World Wide Web. 13–19.Google ScholarDigital Library
- Nina Sakhnini and Debaleena Chattopadhyay. 2022. A Review of Smartphone Fact-Checking Apps and their (Non) Use Among Older Adults. In Adjunct Publication of the 24th International Conference on Human-Computer Interaction with Mobile Devices and Services. 1–8.Google ScholarDigital Library
- Neil Selwyn, Stephen Gorard, John Furlong, and Louise Madden. 2003. Older adults’ use of information and communications technology in everyday life. Ageing & Society 23, 5 (2003), 561–582.Google ScholarCross Ref
- Jack Sidnell and Tanya Stivers. 2012. The handbook of conversation analysis. Vol. 121. John Wiley & Sons.Google Scholar
- Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Thirty-first AAAI conference on artificial intelligence.Google ScholarCross Ref
- Brodrick Stigall and Kelly Caine. 2020. A Systematic Review of Human Factors Literature About Voice User Interfaces and Older Adults. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 64. SAGE Publications Sage CA: Los Angeles, CA, 13–17.Google ScholarCross Ref
- Brodrick Stigall, Jenny Waycott, Steven Baker, and Kelly Caine. 2019. Older adults’ perception and use of voice user interfaces: a preliminary review of the computing literature. In Proceedings of the 31st Australian Conference on Human-Computer-Interaction. 423–427.Google ScholarDigital Library
- Brendan Strahm, Colin M Gray, and Mihaela Vorvoreanu. 2018. Generating mobile application onboarding insights through minimalist instruction. In Proceedings of the 2018 Designing Interactive Systems Conference. 361–372.Google ScholarDigital Library
- Tiffany Tong, Mark Chignell, Mary C Tierney, and Jacques Lee. 2016. A serious game for clinical assessment of cognitive status: validation study. JMIR serious games 4, 1 (2016), e5006.Google Scholar
- Joan Vermeulen, Jacques CL Neyens, Marieke D Spreeuwenberg, Erik van Rossum, Walther Sipers, Herbert Habets, David J Hewson, and Luc P De Witte. 2013. User-centered development and testing of a monitoring system that provides feedback regarding physical functioning to elderly people. Patient preference and adherence 7 (2013), 843.Google Scholar
- Andrea Vianello, Luca Chittaro, Stefano Burigat, and Riccardo Budai. 2017. MotorBrain: a mobile app for the assessment of users’ motor performance in neurology. Computer methods and programs in biomedicine 143 (2017), 35–47.Google Scholar
- John Vines, Stephen Lindsay, Gary W Pritchard, Mabel Lie, David Greathead, Patrick Olivier, and Katie Brittain. 2013. Making family care work: dependence, privacy and remote home monitoring telecare systems. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. 607–616.Google ScholarDigital Library
- Lin Wang, Pei-Luen Patrick Rau, and Gavriel Salvendy. 2011. Older adults’ acceptance of information technology. Educational Gerontology 37, 12 (2011), 1081–1099.Google ScholarCross Ref
- Josh Wardini. 2022. Voice search statistics 2022: Smart speakers, VA, and users: Serpwatch. https://serpwatch.io/blog/voice-search-statistics/Google Scholar
- Jenny Waycott, Frank Vetere, Sonja Pedell, Amee Morgans, Elizabeth Ozanne, and Lars Kulik. 2016. Not for me: Older adults choosing not to participate in a social isolation intervention. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 745–757.Google ScholarDigital Library
- Tzu-Ning Wen, Po-Liang Cheng, and Po-Lun Chang. 2017. Evaluate the usability of the mobile instant messaging software in the elderly. In MEDINFO 2017: Precision Healthcare through Informatics. IOS Press, 818–822.Google Scholar
- Leigh Ann Wilson, Kate O’Loughlin, Robyn Hector, and Deborah Ann Black. 2021. Using a smartphone application (App) to assist older adults to navigate their local area during extreme weather events and changing environmental conditions: A qualitative study. Australasian Journal on Ageing 40, 1 (2021), e95–e99.Google ScholarCross Ref
- Jason Wu, Gabriel Reyes, Sam C White, Xiaoyi Zhang, and Jeffrey P Bigham. 2021. When can accessibility help? an exploration of accessibility feature recommendation on mobile devices. In Proceedings of the 18th International Web for All Conference. 1–12.Google ScholarDigital Library
- Ja Eun Yu and Debaleena Chattopadhyay. 2020. "Maps are hard for me": How Older Adults Struggle with Mobile Maps. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. in press.Google ScholarDigital Library
- Ja Eun Yu and Debaleena Chattopadhyay. 2020. Supporting Older Adults in Locating Mobile Interface Features with Voice Input. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. in press.Google Scholar
- Mingyuan Zhong, Gang Li, Peggy Chi, and Yang Li. 2021. HelpViz: Automatic Generation of Contextual Visual Mobile Tutorials from Text-Based Instructions. In The 34th Annual ACM Symposium on User Interface Software and Technology. 1144–1153.Google ScholarDigital Library
- Jia Zhou, Pei-Luen Patrick Rau, and Gavriel Salvendy. 2012. Use and design of handheld computers for older adults: A review and appraisal. International Journal of Human-Computer Interaction 28, 12(2012), 799–826.Google ScholarCross Ref
Index Terms
- “Where is history”: Toward Designing a Voice Assistant to help Older Adults locate Interface Features quickly
Recommendations
Supporting Older Adults in Locating Mobile Interface Features with Voice Input
ASSETS '20: Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and AccessibilityAs mobile applications continue to offer more features, tackling the complexity of mobile interfaces can become challenging for older adults. Owing to a small screen and frequent updates that modify the visual layouts of menus and buttons, older adults ...
Learnability Assessment of Speech-Based Intelligent Personal Assistants by Older Adults
Human Aspects of IT for the Aged PopulationAbstractOlder adults often have problems and difficulties with adopting technologies but the popularity of technology usage among older adults had increased significantly in recent years. Separately, Speech-Based Intelligent Personal Assistants (sIPA) or ...
Older Adults and Voice Interaction: A Pilot Study with Google Home
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing SystemsIn this paper we present the results of an exploratory study examining the potential of voice assistants (VA) for some groups of older adults in the context of Smart Home Technology (SHT). To research the aspect of older adults' interaction with voice ...
Comments