Shashank Ahire
Michael Rohs
ABSTRACT
Previous literature has reported that users consider hands-free and eyes-free interaction as one of the prime features of IPAs (Intelligent Personal Assistants). Hands-free and eyes-free interaction enables dual tasking. Although users prefer dual tasking with IPAs, it is unknown to what degree current IPAs are compatible with dual tasking. To determine IPA efficiency while dual tasking, we investigate cognitive load in dual-task scenarios with IPAs. In our experiment, we selected a rhythm game as the primary task and everyday IPA requests as secondary tasks. The secondary tasks belonged to four common categories: information search, multimedia control, smart home control, and turn-taking conversations. The findings show that IPAs need significant improvement to support dual tasking. Out of the four categories, only tasks in the smart home and multimedia categories were appropriate for dual tasking, whereas turn-taking conversation and information search had a high cognitive load. Task completion time was significantly different between tasks, but the penalty on the accuracy of the primary task was small. In interviews we found that, due to information abundance in IPA responses and high time pressure during task completion, users tended to make several mistakes. Based on our findings and observations we derive four design recommendations that facilitate dual-tasking while using IPAs.
- Piotr D. Adamczyk and Brian P. Bailey. 2004. If Not Now, When? The Effects of Interruption at Different Moments within Task Execution. Association for Computing Machinery, New York, NY, USA, 271–278. https://doi.org/10.1145/985692.985727Google Scholar
Digital Library
- Shashank Ahire and Michael Rohs. 2020. Tired of Wake Words? Moving Towards Seamless Conversations with Intelligent Personal Assistants. In Proceedings of the 2nd Conference on Conversational User Interfaces (Bilbao, Spain) (CUI ’20). Association for Computing Machinery, New York, NY, USA, Article 20, 3 pages. https://doi.org/10.1145/3405755.3406141Google ScholarDigital Library
- Brian P. Bailey, Joseph A. Konstan, and John V. Carlis. 2001. The Effects of Interruptions on Task Performance, Annoyance, and Anxiety in the User Interface. In Proceedings INTERACT ’01. IOS Press, 593–601.Google Scholar
- Frank Bentley, Chris Luvogt, Max Silverman, Rushani Wirasinghe, Brooke White, and Danielle Lottridge. 2018. Understanding the Long-Term Use of Smart Speaker Assistants. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 3, Article 91 (Sept. 2018), 24 pages. https://doi.org/10.1145/3264901Google ScholarDigital Library
- Narae Cha, Auk Kim, Cheul Young Park, Soowon Kang, Mingyu Park, Jae-Gil Lee, Sangsu Lee, and Uichin Lee. 2020. Hello There! Is Now a Good Time to Talk? Opportune Moments for Proactive Interactions with Smart Speakers. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 3, Article 74 (Sept. 2020), 28 pages. https://doi.org/10.1145/3411810Google ScholarDigital Library
- Minji Cho, Sang-su Lee, and Kun-Pyo Lee. 2019. Once a Kind Friend is Now a Thing: Understanding How Conversational Agents at Home Are Forgotten. In Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA, USA) (DIS ’19). Association for Computing Machinery, New York, NY, USA, 1557–1569. https://doi.org/10.1145/3322276.3322332Google ScholarDigital Library
- Benjamin R. Cowan, Nadia Pantidi, David Coyle, Kellie Morrissey, Peter Clarke, Sara Al-Shehri, David Earley, and Natasha Bandeira. 2017. “What Can i Help You with?”: Infrequent Users’ Experiences of Intelligent Personal Assistants. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (Vienna, Austria) (MobileHCI ’17). Association for Computing Machinery, New York, NY, USA, Article 43, 12 pages. https://doi.org/10.1145/3098279.3098539Google ScholarDigital Library
- Mary Czerwinski, Ed Cutrell, and Eric Horvitz. 2000. Instant Messaging and Interruption: Influence of Task Type on Performance. (December 2000), 356–361. OZCHI 2000 Conference Proceedings.Google Scholar
- Justin Edwards, Christian Janssen, Sandy Gould, and Benjamin R. Cowan. 2021. Eliciting Spoken Interruptions to Inform Proactive Speech Agent Design. In CUI 2021 - 3rd Conference on Conversational User Interfaces (Bilbao (online), Spain) (CUI ’21). Association for Computing Machinery, New York, NY, USA, Article 23, 12 pages. https://doi.org/10.1145/3469595.3469618Google ScholarDigital Library
- Justin Edwards, He Liu, Tianyu Zhou, Sandy J. J. Gould, Leigh Clark, Philip Doyle, and Benjamin R. Cowan. 2019. Multitasking with Alexa: How Using Intelligent Personal Assistants Impacts Language-Based Primary Task Performance. In Proceedings of the 1st International Conference on Conversational User Interfaces (Dublin, Ireland) (CUI ’19). Association for Computing Machinery, New York, NY, USA, Article 4, 7 pages. https://doi.org/10.1145/3342775.3342785Google ScholarDigital Library
- Debjyoti Ghosh, Can Liu, Shengdong Zhao, and Kotaro Hara. 2020. Commanding and Re-Dictation: Developing Eyes-Free Voice-Based Interaction for Editing Dictated Text. ACM Trans. Comput.-Hum. Interact. 27, 4, Article 28 (Aug. 2020), 31 pages. https://doi.org/10.1145/3390889Google ScholarDigital Library
- Mingjun He, Jianbin Guo, and Shengkui Zeng. 2020. Cognitive Load Measurement and Impact Analysis on Performance in Dual-Task Situations. Association for Computing Machinery, New York, NY, USA, 303–307. https://doi.org/10.1145/3425329.3425388Google ScholarDigital Library
- Shamsi T. Iqbal, Yun-Cheng Ju, and Eric Horvitz. 2010. Cars, Calls, and Cognition: Investigating Driving and Divided Attention. Association for Computing Machinery, New York, NY, USA, 1281–1290. https://doi.org/10.1145/1753326.1753518Google ScholarDigital Library
- Myounghoon Jeon, Benjamin K. Davison, Michael A. Nees, Jeff Wilson, and Bruce N. Walker. 2009. Enhanced Auditory Menu Cues Improve Dual Task Performance and Are Preferred with In-Vehicle Technologies(AutomotiveUI ’09). Association for Computing Machinery, New York, NY, USA, 91–98. https://doi.org/10.1145/1620509.1620528Google ScholarDigital Library
- David R. Large, Gary Burnett, Ben Anyasodo, and Lee Skrypchuk. 2016. Assessing Cognitive Demand during Natural Language Interactions with a Digital Driving Assistant. In Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Ann Arbor, MI, USA) (Automotive’UI 16). Association for Computing Machinery, New York, NY, USA, 67–74. https://doi.org/10.1145/3003715.3005408Google ScholarDigital Library
- Terri Lenox, Neil Pilarski, and Lance Leathers. 2012. The Effects of Interruptions on Remembering Task Information. 5 (10 2012).Google Scholar
- Yun Liu, Lu Wang, William R. Kearns, Linda Wagner, John Raiti, Yuntao Wang, and Weichao Yuwen. 2021. Integrating a Voice User Interface into a Virtual Therapy Platform. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3411763.3451595Google ScholarDigital Library
- Ewa Luger and Abigail Sellen. 2016. ”Like Having a Really Bad PA”: The Gulf Between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). ACM, New York, NY, USA, 5286–5297. https://doi.org/10.1145/2858036.2858288Google ScholarDigital Library
- Christine Murad, Cosmin Munteanu, Benjamin R. Cowan, and Leigh Clark. 2019. Revolution or Evolution? Speech Interaction and HCI Design Guidelines. IEEE Pervasive Computing 18, 2 (2019), 33–45. https://doi.org/10.1109/MPRV.2019.2906991Google ScholarDigital Library
- Babette Park and Roland Brünken. 2015. The Rhythm Method: A New Method for Measuring Cognitive Load—An Experimental Dual-Task Study. Applied Cognitive Psychology 29, 2 (2015), 232–243. https://doi.org/10.1002/acp.3100 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/acp.3100Google ScholarCross Ref
- Jennifer Pearson, Simon Robinson, Thomas Reitmaier, Matt Jones, Shashank Ahire, Anirudha Joshi, Deepak Sahoo, Nimish Maravi, and Bhakti Bhikne. 2019. StreetWise: Smart Speakers vs Human Help in Public Slum Settings. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). ACM, New York, NY, USA, Article 96, 13 pages. https://doi.org/10.1145/3290605.3300326Google ScholarDigital Library
- Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice Interfaces in Everyday Life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174214Google ScholarDigital Library
- R. Ratwani, Alyssa E. Andrews, Jenny D. Sousk, and J. Trafton. 2008. The Effect of Interruption Modality on Primary Task Resumption. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 52(2008), 393 – 397.Google ScholarCross Ref
- Simon Robinson, Jennifer Pearson, Shashank Ahire, Rini Ahirwar, Bhakti Bhikne, Nimish Maravi, and Matt Jones. 2018. Revisiting “Hole in the Wall” Computing: Private Smart Speakers and Public Slum Settings. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). ACM, New York, NY, USA, Article 498, 11 pages. https://doi.org/10.1145/3173574.3174072Google ScholarDigital Library
- Ben F. Rubin and Ry Crist. 2019. What Amazon’s Alexa will tell us in 20191. Retrieved September 6 2021 from https://www.cnet.com/home/smart-home/what-amazon-alexa-will-tell-us-in-2019/Google Scholar
- Dario D. Salvucci, Daniel Markley, Mark Zuber, and Duncan P. Brumby. 2007. IPod Distraction: Effects of Portable Music-Player Use on Driver Performance(CHI ’07). Association for Computing Machinery, New York, NY, USA, 243–250. https://doi.org/10.1145/1240624.1240665Google ScholarDigital Library
- Ben Sawyer, Joonbum Lee, Jonathan Dobres, Bruce Mehler, Joseph Coughlin, and Bryan Reimer. 2016. Effects of a Voice Interface on Mirror Check Decrements in Older and Younger Multitasking Drivers. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 60 (09 2016), 16–20. https://doi.org/10.1177/1541931213601004Google ScholarCross Ref
- ERIC HAL SCHWARTZ. 2019. LifePod is Taking Reservations for ‘Proactive’ Voice Assistant and Smart Speaker for the Elderly. voiceBot. Retrieved September 6 2021 from https://voicebot.ai/2019/06/27/lifepod-is-taking-reservations-for-proactive-voice-assistant-and-smart-speaker-for-the-elderly/Google Scholar
- Alex Sciuto, Arnita Saini, Jodi Forlizzi, and Jason I. Hong. 2018. ”Hey Alexa, What’s Up?”: A Mixed-Methods Studies of In-Home Conversational Agent Usage. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 857–868. https://doi.org/10.1145/3196709.3196772Google ScholarDigital Library
- Katie Seaborn and Jacqueline Urakami. 2021. Measuring Voice UX Quantitatively: A Rapid Review. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3411763.3451712Google ScholarDigital Library
- David Strayer, Joel Cooper, Jonna Turrill, James Coleman, and Rachel Hopman. 2016. Talking to your car can drive you to distraction. Cognitive Research: Principles and Implications 1 (12 2016). https://doi.org/10.1186/s41235-016-0018-3Google ScholarCross Ref
- David Strayer, Joel Cooper, Jonna Turrill, James Coleman, and Rachel Hopman. 2017. The smartphone and the driver’s cognitive workload: A comparison of Apple, Google, and Microsoft’s intelligent personal assistants. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale 71 (06 2017), 93–110. https://doi.org/10.1037/cep0000104Google ScholarCross Ref
- Bernhard Suhm. 2003. Towards best practices for speech user interface design.Google Scholar
- Bernhard Suhm, Brad Myers, and Alex Waibel. 2001. Multimodal Error Correction for Speech User Interfaces. ACM Trans. Comput.-Hum. Interact. 8, 1 (March 2001), 60–98. https://doi.org/10.1145/371127.371166Google ScholarDigital Library
- Milka Trajkova and Aqueasha Martin-Hammond. 2020. ”Alexa is a Toy”: Exploring Older Adults’ Reasons for Using, Limiting, and Abandoning Echo. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376760Google ScholarDigital Library
- Jing Wei, Tilman Dingler, and Vassilis Kostakos. 2021. Developing the Proactive Speaker Prototype Based on Google Home. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3411763.3451642Google ScholarDigital Library
- Zhuxiaona Wei and James A. Landay. 2018. Evaluating Speech-Based Smart Devices Using New Usability Heuristics. IEEE Pervasive Computing 17, 2 (2018), 84–96. https://doi.org/10.1109/MPRV.2018.022511249Google ScholarDigital Library
- Wikipedia contributors. 2019. NASA-TLX — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=NASA-TLX&oldid=916786772 [Online; accessed 24-February-2021].Google Scholar
- Wikipedia contributors. 2021. Osu! — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=Osu!&oldid=1007577903 [Online; accessed 21-February-2021].Google Scholar
- Yunhan Wu, Justin Edwards, Orla Cooney, Anna Bleakley, Philip R. Doyle, Leigh Clark, Daniel Rough, and Benjamin R. Cowan. 2020. Mental Workload and Language Production in Non-Native Speaker IPA Interaction. In Proceedings of the 2nd Conference on Conversational User Interfaces (Bilbao, Spain) (CUI ’20). Association for Computing Machinery, New York, NY, USA, Article 3, 8 pages. https://doi.org/10.1145/3405755.3406118Google ScholarDigital Library
Index Terms
- How Compatible is Alexa with Dual Tasking? — Towards Intelligent Personal Assistants for Dual-Task Situations
Index terms have been assigned to the content through auto-classification.
Recommendations
Cognitive Load Measurement and Impact Analysis on Performance in Dual-task Situations
WSSE '20: Proceedings of the 2nd World Symposium on Software EngineeringAlong with the development of automation technology, the focus on human-machine interaction has gradually shifted from physical interaction to cognitive interaction. Especially, the cognitive load measurement is important to pilots who are up against ...
Multitasking with Alexa: how using intelligent personal assistants impacts language-based primary task performance
CUI '19: Proceedings of the 1st International Conference on Conversational User InterfacesIntelligent personal assistants (IPAs) are supposed to help us multitask. Yet the impact of IPA use on multitasking is not clearly quantified, particularly in situations where primary tasks are also language based. Using a dual task paradigm, our study ...
Using stroop task to assess cognitive load
ECCE '10: Proceedings of the 28th Annual European Conference on Cognitive ErgonomicsMotivation -- Assessment of cognitive load on user tasks is useful for characterizing user interfaces and tasks with respect to their demands on the user's mental effort.
Research approach -- We conducted a controlled experiment with 48 subjects. The ...
Comments