skip to main content
10.1145/3297280.3299746acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

Cognitive workload of in-car auditory-vocal interfaces on visuospatial sketchpad based on a dual task of visual pattern test

Published: 08 April 2019 Publication History

Abstract

The goal of this study is to quantify the cognitive workload of visuospatial components on operating voice-based interfaces. Particularly, we aim to quantify the user's visuospatial workload when they operate voice commands while driving and, then, comparing the reported workload that participants simultaneously used graphical interfaces. We used the quantitative measurement method to evaluate the workload on the visuospatial sketchpad by employing a dual task of pattern span test and usage of a target interface. The results indicated that even voice command interfaces affect the performance of the pattern span test regardless of the independence of the sketchpad and the phonological loop. Also, we quantitatively found that employing familiar words and their combinations for drivers could reduce the workload of voice-based operations.

References

[1]
Distraction | National Highway Traffic Safety Administration (NHTSA). http://www.nhtsa.gov/Research/Human+Factors/Distraction. http://www.nhtsa.gov/Research/Human+Factors/Distraction
[2]
HARDIE Design Guidelines Handbook: Human Factors Guidelines for Information Presentation by ATT Systems. 1994. http://www.umich.edu/~driving/guidelines/HardieGuidelinesChapters(old)/HARDIE_Guidelines(Intro).pdf
[3]
The Balanced Corpus of Contemporary Written Japanese (BCCWJ). 2011. http://pj.ninjal.ac.jp/corpus_center/bccwj/en/
[4]
Alan Baddeley. 2000. The episodic buffer: a new component of working memory? Trends in cognitive sciences 4, 11 (2000), 417--423.
[5]
Alan Baddeley. 2007. Working Memory, Thought, and Action. Oxford University Press.
[6]
Luca Chittaro and Luca De Marco. 2004. Driver distraction caused by mobile devices: studying and reducing safety risks. In Proceedings of the 1st Int'l Workshop Mobile Technologies and Health: Benefits and Risks (Udine, Italy, 2004).
[7]
Lewis L Chuang, Christiane Glatz, and Stas Krupenia. 2017. Using EEG to understand why behavior to auditory in-vehicle notifications differs across test environments. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, 123--133.
[8]
Jacob Cohen. 1992. A power primer. Psychological bulletin 112, 1 (1992), 155.
[9]
Nelson Cowan. 2017. The many faces of working memory and short-term storage. Psychonomic bulletin & review 24, 4 (2017), 1158--1170.
[10]
Meredyth Daneman and Patricia A Carpenter. 1980. Individual differences in working memory and reading. Journal of verbal learning and verbal behavior 19, 4 (1980), 450--466.
[11]
S. Delia Sala, C. Gray, A. Baddeley, N. Allamano, and L. Wilson. 1999. Pattern span: a tool for unwelding visuo-spatial memory. Neuropsychologia 37, 10 (1999), 1189--1199.
[12]
ISO Working Draft. 2004. Road vehicles---Ergonomic aspects of transport information and control systems---Simulated lane change test to assess driver distraction. ISO TC 22 (2004).
[13]
Naomi P Friedman and Akira Miyake. 2004. The relations among inhibition and interference control functions: a latent-variable analysis. Journal of experimental psychology: General 133, 1 (2004), 101.
[14]
Susan Gathercole and Tracy Packiam Alloway. 2008. Working memory and learning: A practical guide for teachers. Sage.
[15]
Susan E Gathercole, Tracy P Alloway, Hannah J Kirkwood, Julian G Elliott, Joni Holmes, and Kerry A Hilton. 2008. Attentional and executive function behaviours in children with poor working memory. Learning and individual differences 18, 2 (2008), 214--223.
[16]
P. Green, W. Levison, G. Paelke, and C. Serafin. 1993. Suggested Human Factors Design Guidelines for Driver Information Systems. Technical Report UMTRI-93-21 The University of Michigan Transportation Research Institute (1993).
[17]
Shigeru Haga and Naoki Mizukami. 1996. Japanese version of NASA Task Load Index: Sensitivity of its workload score to difficulty of three different laboratory tasks. The Japanese Journal of Ergonomics 32, 2 (1996), 71--79.
[18]
Sandra G Hart. 2006. NASA-task load index (NASA-TLX); 20 years later. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 50. Sage Publications, 904--908.
[19]
Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology 52 (1988), 139--183.
[20]
inc. (JAMA) Japan automobile manufacturers association. 2004. Guidelines for In-vehicle Display Systems Version 3.0. http://www.jama.or.jp/safe/guideline/pdf/jama_guidelines_v30_en.pdf
[21]
Dylan M Jones, William J Macken, and Alastair P Nicholls. 2004. The phonological store of working memory: Is it phonological and is it a store? Journal of Experimental Psychology: Learning, Memory, and Cognition 30, 3 (2004), 656.
[22]
Wayne K Kirchner. 1958. Age differences in short-term retention of rapidly changing information. Journal of experimental psychology 55, 4 (1958), 352.
[23]
Thomas Kuhn, Akhtar Jameel, M Stumpfle, and Afsaneh Haddadi. 1999. Hybrid in-car speech recognition for mobile multimedia applications. In Vehicular Technology Conference, 1999 IEEE 49th, Vol. 3. IEEE, 2009--2013.
[24]
Tuomo Kujala and Hilkka Grahn. 2017. Visual Distraction Effects of In-Car Text Entry Methods: Comparing Keyboard, Handwriting and Voice Recognition. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, 1--10.
[25]
Bonnie M Lawrence, Joel Myerson, and Richard A Abrams. 2004. Interference with spatial working memory: An eye movement is more than a shift of attention. Psychonomic Bulletin & Review 11, 3 (2004), 488--494.
[26]
John D Lee, Daniel V McGehee, Timothy L Brown, and Michelle L Reyes. 2002. Collision warning timing, driver distraction, and driver response to imminent rear-end collisions in a high-fidelity driving simulator. Human Factors: The Journal of the Human Factors and Ergonomics Society 44, 2 (2002), 314--334.
[27]
Stephan Lewandowsky. 2016. #whatWM? A digital event celebrating the 9 lives of working memory. https://featuredcontent.psychonomic.org/whatwm-a-digital-event-celebrating-the-9-lives-of-working-memory/.
[28]
Robert H Logie. 2014. Visuo-spatial working memory. Psychology Press.
[29]
Thomas B Martin. 1976. Practical applications of voice input to machines. Proc. IEEE 64, 4 (1976), 487--501.
[30]
Jennifer C McVay and Michael J Kane. 2009. Conducting the train of thought: working memory capacity, goal neglect, and mind wandering in an executive-control task. Journal of Experimental Psychology: Learning, Memory, and Cognition 35, 1 (2009), 196.
[31]
Bruce Mehler, Bryan Reimer, Jonathan Dobres, Hale McAnulty, Alea Mehler, Daniel Munger, and Joseph F Coughlin. 2014. Further evaluation of the effects of a production level "voice-command" interface on driver behavior: replication and a consideration of the significance of training method. MIT AgeLab Technical Report (2014).
[32]
Takahiro Miura, Ken-ichiro Yabu, Kenichi Tanaka, Hiroshi Ozawa, Masamitsu Furukawa, Seiko Michiyoshi, Tetsuya Yamamoto, Kazutaka Ueda, and Tohru Ifukube. 2016. Visuospatial Workload Measurement of an Interface Based on a Dual Task of Visual Working Memory Test. In Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Automotive'UI 16). ACM, New York, NY, USA, 9--17.
[33]
Takahiro Miura, Ken-ichiro Yabu, Kenichi Tanaka, Kazutaka Ueda, and Tohru Ifukube. 2017. VisuoSpats: A Gamified Application to Measure Visuospatial Working Memory Volume. ITE Transactions on Media Technology and Applications 5, 1 (2017), 8--16.
[34]
Akira Miyake, Naomi P Friedman, David A Rettinger, Priti Shah, and Mary Hegarty. 2001. How are visuospatial working memory, executive functioning, and spatial abilities related? A latent-variable analysis. Journal of experimental psychology: General 130, 4 (2001), 621.
[35]
Stephen Oh, Vishu Viswanathan, and Panos Papamichalis. 1992. Hands-free voice communication in an automobile with a microphone array. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on, Vol. 1. IEEE, 281--284.
[36]
Dennis Orth, Nadja Schömig, Christian Mark, Monika Jagiellowicz-Kaufmann, Dorothea Kolossa, and Martin Heckmann. 2017. Benefits of Personalization in the Context of a Speech-Based Left-Turn Assistant. submitted to Automotive User Interfaces (2017).
[37]
Bastian Pfleging, Stefan Schneegass, and Albrecht Schmidt. 2012. Multimodal interaction in the car: combining speech and gestures on the steering wheel. In Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, 155--162.
[38]
Susan J. Pickering. 2001. The development of visuo-spatial working memory. Memory 9, 4-6 (2001), 423--432.
[39]
Roberto Pieraccini, Krishna Dayanidhi, Jonathan Bloom, Jean-GuiDahan, Michael Phillips, Bryan R Goodman, and K Venkatesh Prasad. 2004. Multimodal conversational systems for automobiles. Commun. ACM 47, 1 (2004), 47--49.
[40]
Leah M Reeves, Jennifer Lai, James A Larson, Sharon Oviatt, TS Balaji, Stéphanie Buisine, Penny Collings, Phil Cohen, Ben Kraal, Jean-Claude Martin, et al. 2004. Guidelines for multimodal user interface design. Commun. ACM 47, 1 (2004), 57--59.
[41]
Bryan Reimer, Bruce Mehler, J Dobres, and JF Coughlin. 2013. The effects of a production level "voice-command" interface on driver behavior: summary findings on reported workload, physiology, visual attention, and driving performance.
[42]
Bryan Reimer, Bruce Mehler, Jonathan Dobres, Hale McAnulty, Alea Mehler, Daniel Munger, and Adrian Rumpold. 2014. Effects of an'Expert Mode'voice command system on task performance, glance behavior & driver physiology. In Proceedings of the 6th international conference on automotive user interfaces and interactive vehicular applications. ACM, 1--9.
[43]
Virginia M Rosen and Randall W Engle. 1997. The role of working memory capacity in retrieval. Journal of Experimental Psychology: General 126, 3 (1997), 211.
[44]
Satoru Suto and Muneyoshi Hyodo. 2006. Visual rehearsal within visuo-spatial working memory: A comparison of recall and recognition in a visual pattern test. The Japanese Journal of Cognitive Psychology (2006), 149--156.
[45]
Yukio Tsuchida. 2009. What is working memory capacity?: Individual difference and influence on cognitive performance. Bulletin of Faculty of Education, Hokkaido University 109 (2009), 81--92.

Index Terms

  1. Cognitive workload of in-car auditory-vocal interfaces on visuospatial sketchpad based on a dual task of visual pattern test

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        SAC '19: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing
        April 2019
        2682 pages
        ISBN:9781450359337
        DOI:10.1145/3297280
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 08 April 2019

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. driver distraction
        2. in-vehicle information system
        3. visuospatial sketchpad
        4. voice command operations
        5. working memory

        Qualifiers

        • Research-article

        Conference

        SAC '19
        Sponsor:

        Acceptance Rates

        Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

        Upcoming Conference

        SAC '25
        The 40th ACM/SIGAPP Symposium on Applied Computing
        March 31 - April 4, 2025
        Catania , Italy

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • 0
          Total Citations
        • 199
          Total Downloads
        • Downloads (Last 12 months)18
        • Downloads (Last 6 weeks)2
        Reflects downloads up to 15 Feb 2025

        Other Metrics

        Citations

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media