Skip to main content
SpringerLink
Log in

A wearable virtual touch system for IVIS in cars

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

In automotive domain, operation of secondary tasks like accessing infotainment system, adjusting air conditioning vents, and side mirrors distract drivers from driving. Though existing modalities like gesture and speech recognition systems facilitate undertaking secondary tasks by reducing duration of eyes off the road, those often require remembering a set of gestures or screen sequences. In this paper, we have proposed two different modalities for drivers to virtually touch the dashboard display using a laser tracker with a mechanical switch and an eye gaze switch. We compared performances of our proposed modalities against conventional touch modality in automotive environment by comparing pointing and selection times of representative secondary task and also analysed effect on driving performance in terms of deviation from lane, average speed, variation in perceived workload and system usability. We did not find significant difference in driving and pointing performance between laser tracking system and existing touchscreen system. Our result also showed that the driving and pointing performance of the virtual touch system with eye gaze switch was significantly better than the same with mechanical switch. We evaluated the efficacy of the proposed virtual touch system with eye gaze switch inside a real car and investigated acceptance of the system by professional drivers using qualitative research. The quantitative and qualitative studies indicated importance of using multimodal system inside car and highlighted several criteria for acceptance of new automotive user interface.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

References

  1. Adell E (2009) Driver experience and acceptance of driver support systems-a case of speed adaptation. Lund University 125(126):148

    Google Scholar 

  2. Aguilar SR, Merino JLM, Sánchez AM, Valdivieso ÁS (2015) Variation of the heartbeat and activity as an indicator of drowsiness at the wheel using a smartwatch. Int J Artif Intell Interact Multimedia 3

  3. Ahmad BI, Langdon PM, Godsill SJ, Hardy R, Dias E, Skrypchuk L (2014) Interactive displays in vehicles: Improving usability with a pointing gesture tracker and Bayesian intent predictors. In proceedings of the 6th international conference on automotive user interfaces and interactive vehicular applications (pp. 1–8). ACM

  4. Ahmad BI, Langdon PM, Godsill SJ, Donkor R, Wilde R, Skrypchuk L (2016) You do not have to touch to select: a study on predictive in-car touchscreen with mid-air selection. In proceedings of the 8th international conference on automotive user interfaces and interactive vehicular applications (pp. 113–120). ACM

  5. Amoura C, Berjot S, Gillet N, Altintas E (2014) Desire for control, perception of control: their impact on autonomous motivation and psychological adjustment. Motiv Emot 38(3):323–335

    Article  Google Scholar 

  6. [Ayata 2018] Ayata, D., Yaslan, Y., & Kamasak, M. E. (2018). Emotion Based Music Recommendation System Using Wearable Physiological Sensors. IEEE Transactions on Consumer Electronics.

  7. Baguley T, Andrews M (2016) Handling missing data. In: Robertson J, Kaptein M (eds) Modern statistical methods for HCI. Springer, pp 57–82

    Chapter  Google Scholar 

  8. Biswas P, Roy S, Prabhakar, G, Rajesh J, Arjun S, Arora M, Gurumoorthy B, Chakrabarti A, Interactive sensor visualization for smart manufacturing system, proceedings of the 31st British human computer interaction conference 2017 (British HCI 17)

  9. Biswas P, Aydemir GA, Langdon P, Godsill S (2013) Intent recognition using neural networks and Kalman filters. In Human-computer interaction and knowledge discovery in complex, unstructured, Big Data. Springer, Berlin, Heidelberg, pp. 112–123

  10. Biswas P, Langdon P (2014) Multimodal target prediction model. In CHI'14 Extended abstracts on human factors in computing systems. ACM pp. 1543–1548

  11. Biswas P, Langdon P (2015) Multimodal intelligent eye-gaze tracking system. Int J Human Comput Interact 31(4):277–294

    Article  Google Scholar 

  12. Biswas P, Prabhakar, G, Rajesh J, Pandit K, Halder A (2017) Improving eye gaze controlled car dashboard using simulated annealing. In Proceedings of the 31st British computer society human computer interaction conference (p. 39). BCS Learning & Development Ltd

  13. Chang W, Hwang W, Ji YG (2011) Haptic seat interfaces for driver information and warning systems. Int J Human Comput Interact 27(12):1119–1132

    Article  Google Scholar 

  14. Corbin J (2015) Basics of qualitative research. Sage Publications

    Google Scholar 

  15. Debnath A, Kobra KT, Rawshan PP, Paramita M, Islam MN (2018) An explication of acceptability of wearable devices in context of bangladesh: a user study. In 2018 IEEE 6th international conference on future internet of things and cloud (FiCloud). IEEE pp. 136–140

  16. Dey P, Paul A, Saha D, Mukherjee S, Nath A (2012) Laser beam operated windows operation. In 2012 international conference on communication systems and network technologies. IEEE pp. 594–599

  17. Fitts PM (1954) The information capacity of the human motor system in controlling the amplitude of movement. J Exp Psychol 47(6):381

    Article  Google Scholar 

  18. Ganz A, Schafer JM, Tao Y, Wilson C, Robertson M (2014) PERCEPT-II: smartphone based indoor navigation system for the blind, 2014 36th annual international conference of the IEEE engineering in medicine and biology society, Chicago, IL, USA, pp. 3662-3665, https://doi.org/10.1109/EMBC.2014.6944417

  19. Gorlewicz JL, Tennison JL, Uesbeck PM, Richard ME, Palani HP, Stefik A, Smith DW, Giudice NA (2020) Design guidelines and recommendations for multimodal, touchscreen-based graphics. ACM Trans Access Comput (TACCESS) 13(3):1–30

  20. Khan WM, Zualkernan IA (2018) SensePods: a zigbee-based tangible smart home interface. In: IEEE transactions on consumer electronics, vol 64, no. 2. pp 145–152. https://doi.org/10.1109/TCE.2018.2844729

  21. Kern D, Schmidt A (2009) Design space for driver-based automotive user interfaces. In Proceedings of the 1st international conference on automotive user interfaces and interactive vehicular applications (AutomotiveUI '09). Association for computing machinery, New York, NY, USA, 3–10. https://doi.org/10.1145/1620509.1620511

  22. Kim JH, Lim JH, Jo CI, Kim K (2015) Utilization of visual information perception characteristics to improve classification accuracy of driver’s visual search intention for intelligent vehicle. Int J Human Comput Interact 31(10):717–729

    Article  Google Scholar 

  23. Kundinger T, Yalavarthi PK, Riener A, Wintersberger P, Schartmüller C (2020) Feasibility of smart wearables for driver drowsiness detection and its potential among different age groups. Int J Pervasive Comput Commun 16(1)

  24. Lank E, Cheng YCN, Ruiz J (2007) Endpoint prediction using motion kinematics. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, ACM, pp. 637–646

  25. Liang Y, Reyes ML, Lee JD (2007) Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans Intell Transp Syst 8:340–350

    Article  Google Scholar 

  26. Mattes S (2003) The lane-change-task as a tool for driver distraction evaluation. Qual Work Prod Enterp Future 57:60

    Google Scholar 

  27. Merriam-Webster. (n.d.). Retrieved July 24, 2020 from www.merriam-webster.com: https://www.merriamwebster.com/dictionary/purchasing%20power

  28. Mulloni A, Seichter H, Schmalstieg D (2011) Handheld augmented reality indoor navigation with activity-based instructions. In Proceedings of the 13th international conference on human computer interaction with mobile devices and services (MobileHCI '11). Association for Computing Machinery, New York, NY, USA, 211–220

  29. Murata A (1998) Improvement of pointing time by predicting targets in pointing with a PC mouse. Int J Human Comput Interact 10(1):23–32

    Article  Google Scholar 

  30. NHTSA (2012) Visual-Manual NHTSA driver distraction guidelines for in-vehicle electronic devices: notice of proposed federal guidelines. Fed Reg 77(37):11199–11250

    Google Scholar 

  31. Nordhoff S, De Winter J, Kyriakidis M, Van Arem B, Happee R (2018) Acceptance of driverless vehicles: results from a large cross-national questionnaire study. J Adv Transp 2018

  32. Normark CJ (2015) Design and evaluation of a touch-based personalizable in-vehicle user interface. Int J Human Comput Interact 31(11):731–745

    Article  Google Scholar 

  33. Ohn-Bar E, Trivedi MM (2014) Hand gesture recognition in real time for automotive interfaces: a multimodal vision-based approach and evaluations. IEEE Trans Intell Transp Syst 15(6):2368–2377

    Article  Google Scholar 

  34. Palani HP, Fink PD, Giudice NA (2020) Design guidelines for schematizing and rendering haptically perceivable graphical elements on touchscreen devices. Int J Human Comput Interact 36(15):1393–1414

  35. Pasqual PT, Wobbrock JO (2014) Mouse pointing endpoint prediction using kinematic template matching. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp. 743–752

  36. Prabhakar G, Rajesh J, Biswas P (2016) Comparison of three hand movement tracking sensors as cursor controllers. In Control, instrumentation, communication and computational technologies (ICCICCT), 2016 International Conference on. IEEE pp. 358–364

  37. Prabhakar G, Biswas P (2017) Evaluation of laser pointer as a pointing device in automotive. In 2017 international conference on intelligent computing, instrumentation and control technologies (ICICICT). IEEE pp. 364–371

  38. Prabhakar G, Ramakrishnan A, Murthy LRD, Sharma VK, Madan M, Deshmukh S, Biswas P (2019) Interactive Gaze & finger controlled HUD for Cars. J Multimod User Interf 14:101–121

  39. Rocha S, Lopes A (2020) Navigation based application with augmented reality and accessibility. In Extended abstracts of the 2020 CHI conference on human factors in computing systems (CHI EA '20). Association for computing machinery, New York, NY, USA, 1–9

  40. Schmidtler J, Bengler K, Dimeas F, Campeau-Lecours A (2017) A questionnaire for the evaluation of physical assistive devices (quead): testing usability and acceptance in physical human-robot interaction. In 2017 IEEE international conference on systems, man, and cybernetics (SMC). IEEE pp. 876–881

  41. Schnelle-Walka D, Radomski S (2019) Automotive multimodal human-machine interface. In: The handbook of multimodal-multisensor interfaces: language processing, software, commer- cialization, and emerging directions, vol 3. pp 477–522

  42. Spagnolli A, Guardigli E, Orso V, Varotto A, Gamberini L (2015) Measuring user acceptance of wearable symbiotic devices: validation study across application scenarios. In International workshop on symbiotic interaction. Springer, Cham, pp. 87–98

  43. Steinberger F, Schroeter R, Babiac D (2017) Engaged drivers–safe drivers: gathering real-time data from mobile and wearable devices for safe-driving apps. In Automotive user interfaces. Springer, Cham, pp. 55–76

  44. Stern RM, Ray WJ, Quigley KS (2001) Psychophysiological recording. Oxford University Press

    Google Scholar 

  45. San Vito PDC, Shakeri G, Brewster SA, Pollick FE, Brown E, Skrypchuk L, Mouzakitis A (2019) Haptic Navigation Cues On The Steering Wheel. In CHI (p. 210)

  46. Weinberg G, Knowles A, Langer P (2012) Bullseye: an automotive touch interface that’s always on target. In Adjunct! Proceedings!. p. 43

  47. Witkowski Todd R, Kurt A Dykema, Steven L Geerlings, Mark L Zeinstra, Robert F Buege (2014) Wireless control system and method. U.S. Patent 8,634,888, issued January 21

  48. Woelfl G (2020) U.S. Patent No. 10,674,268. Washington, DC: U.S. Patent and Trademark Office

  49. Woodworth RS (1899) The accuracy of voluntary movement. Psychol Revi, pp. 1–119

  50. Yerkes RM, Dodson JD (1908) The relation of strength of stimulus to rapidity of habit formation. The J Comp Neurol 27–41

  51. Zhang Y, Lin WC, Chin YKS (2010) A pattern-recognition approach for driving skill characterization. IEEE Trans Intell Transp Syst 11:905–916

    Article  Google Scholar 

  52. Ziebart BD (2010) Modeling purposeful adaptive behavior with the principle of maximum causal entropy

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Science, Bangalore, 560012, India

    Gowdham Prabhakar, Priyam Rajkhowa & Pradipta Biswas

  2. Indian Institute of Information Technology Design and Manufacturing, Kancheepuram, 600127, India

    Dharmesh Harsha

Authors
  1. Gowdham Prabhakar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Priyam Rajkhowa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dharmesh Harsha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pradipta Biswas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pradipta Biswas.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (MP4 54508 kb)

Supplementary file2 (MP4 34235 kb)

Supplementary file3 (DOCX 22 kb)

Supplementary file4 (DOCX 35 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Prabhakar, G., Rajkhowa, P., Harsha, D. et al. A wearable virtual touch system for IVIS in cars. J Multimodal User Interfaces 16, 87–106 (2022). https://doi.org/10.1007/s12193-021-00377-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-021-00377-9

Keywords

Search

Navigation