Skip to main content
SpringerLink
Log in

Investigating the effects of modality switches on driver distraction and interaction efficiency in the car

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

In the last decade, the number and variety of secondary tasks in modern vehicles has grown exponentially. To address this variety, drivers can choose between alternative input modalities to complete each task in the most adequate way. However, the process of switching between different modalities might cause increased cognitive effort and finally result in a loss of efficiency. Therefore, the effects of switching between input modalities have to be examined in detail. We present a user study with 18 participants that investigates these effects when switching between touch and speech input on task efficiency and driver distraction in a dual-task setup. Our results show that the sequential combination of adequate modalities for subtasks did not affect task completion times and thus reduced the duration of the entire interaction. We argue to promote modality switches and discuss the implications on application areas beyond the automotive context.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bradford JH, James H (1995) The human factors of speech-based interfaces. ACM SIGCHI Bull 27(2):61–67. https://doi.org/10.1145/202511.202527

    Article  Google Scholar 

  2. Card SK, Moran TP, Newell A (1980) The keystroke-level model for user performance time with interactive systems. Commun ACM 23(7):396–410. https://doi.org/10.1145/358886.358895

    Article  Google Scholar 

  3. Gondan M, Lange K, Rösler F, Röder B (2004) The redundant target effect is affected by modality switch costs. Psychon Bull Rev 11(2):307–313

    Article  Google Scholar 

  4. Harbluk JL, Noy YI, Trbovich PL, Eizenman M (2007) An on-road assessment of cognitive distraction: impacts on drivers visual behavior and braking performance. Accid Anal Prev 39(2):372–379. https://doi.org/10.1016/j.aap.2006.08.013

    Article  Google Scholar 

  5. Koons DB, Sparrell CJ, Thorisson KR (1993) Integrating simultaneous input from speech, gaze, and hand gestures. In: Maybury MT (ed) Intelligent multimedia interfaces. AAAI Press, Menlo Park, pp 257–276

    Google Scholar 

  6. Mitrevska M, Moniri MM, Nesselrath R, Schwartz T, Feld M, Korber Y, Deru M, Muller C (2015) SiAM—situation-adaptive multimodal interaction for innovative mobility concepts of the future. In: Proceedings of the international conference on intelligent environments—IE ’15. IEEE, pp 180–183. https://doi.org/10.1109/IE.2015.39

  7. Monsell S (2003) Task switching. Trends Cogn Sci 7(3):134–140. https://doi.org/10.1016/S1364-6613(03)00028-7

    Article  Google Scholar 

  8. Müller C, Weinberg G, Vetro A (2011) Multimodal input in the car, today and tomorrow. IEEE Multimed 18(1):98–103. https://doi.org/10.1109/MMUL.2011.14

    Article  Google Scholar 

  9. Ohn-Bar E, Trivedi MM (2014) Hand gesture recognition in real time for automotive interfaces: a multimodal vision-based approach and evaluations. IEEE Trans Intell Transp Syst 15(6):2368–2377. https://doi.org/10.1109/TITS.2014.2337331

    Article  Google Scholar 

  10. Oviatt S (2012) Multimodal interfaces. In: Jacko JA (ed) The human–computer interaction handbook: fundamentals, evolving technologies and emerging applications, 14th edn. CRC Press, Boca Raton, pp 286–304

    Google Scholar 

  11. Oviatt S, Coulston R, Lunsford R (2004) When do we interact multimodally? Cognitive load and multimodal communication patterns. In: Proceedings of the 2004 international conference on multimodal interfaces—ICMI ’04, pp 129–136. http://doi.acm.org/10.1145/1027933.1027957

  12. Petzoldt T, Bellem H, Krems JF (2014) The critical tracking task: a potentially useful method to assess driver distraction? Hum Fact J Hum Fact Ergonom Soc 56(4):789–808. https://doi.org/10.1177/0018720813501864

    Article  Google Scholar 

  13. Pfleging B, Schneegass S, Schmidt A (2012) Multimodal interaction in the car—combining speech and gestures on the steering wheel. In: Proceedings of the 4th international conference on automotive user interfaces and interactive vehicular applications—AutomotiveUI ’12. ACM, New York, pp 0–7. https://doi.org/10.1145/2390256.2390282

  14. Pickering CA, Burnham KJ, Richardson MJ (2007) A research study of hand gesture recognition technologies and applications for human vehicle interaction. In: Institution of engineering and technology conference on automotive electronics, pp 1–15

  15. Pickering CA, Burnham KKJ, Richardson MJM (2007) A review of automotive human machine interface technologies and techniques to reduce driver distraction. In: 2nd IET international conference on system safety. IEEE, pp 223–228. https://doi.org/10.1049/cp:20070468

  16. Roider F, Rümelin S, Gross T (2018) Using visual cues to leverage the use of speech input in the vehicle. In: International conference on persuasive technology. Springer, Cham, pp 120–131. https://doi.org/10.1007/978-3-319-78978-1_10

  17. Schneegaß S, Pfleging B, Kern D, Schmidt A (2011) Support for modeling interaction with automotive user interfaces. In: Proceedings of the 3rd international conference on automotive user interfaces and interactive vehicular applications—AutomotiveUI ’11, p 71. https://doi.org/10.1145/2381416.2381428

  18. Strayer DL, Watson JM, Drews FA (2011) Cognitive distraction while multitasking in the automobile. In: Federmeier KD (ed) Psychology of learning and motivation, vol 54. Elsevier, Amsterdam. https://doi.org/10.1016/B978-0-12-385527-5.00002-4

    Google Scholar 

  19. Wickens CD, Sandry DL, Vidulich M (1983) Compatibility and resource competition between modalities of input, central processing, and output. Hum Fact 25(2):227–248. https://doi.org/10.1177/001872088302500209

    Article  Google Scholar 

  20. Wierwille WW (1993) Demands on driver resources associated with introducing advanced technology into the vehicle. Transp Res Part C Emerg Technol 1(2):133–142. https://doi.org/10.1016/0968-090X(93)90010-D

    Article  Google Scholar 

  21. Zhang Y, Stellmach S, Sellen A (2015) The costs and benefits of combining gaze and hand gestures for remote. In: Abascal J, Barbosa S, Fetter M, Gross T, Palanque P, Winckler M (eds) Human–computer interaction—INTERACT 2015. INTERACT 2015. Lecture notes in computer science. Springer, Cham. https://doi.org/10.1007/978-3-319-22698-9

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. BMW Group Research, New Technologies Innovation, Parkring 19, 85478, Garching bei München, Germany

    Florian Roider & Sonja Rümelin

  2. Universität Bamberg, Bamberg, Germany

    Florian Roider & Tom Gross

  3. Ludwig-Maximilians-Universität München, Munich, Germany

    Bastian Pfleging

Authors
  1. Florian Roider
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sonja Rümelin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bastian Pfleging
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tom Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Florian Roider.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roider, F., Rümelin, S., Pfleging, B. et al. Investigating the effects of modality switches on driver distraction and interaction efficiency in the car. J Multimodal User Interfaces 13, 89–97 (2019). https://doi.org/10.1007/s12193-019-00297-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-019-00297-9

Keywords

Search

Navigation