Alvin Jude
G. Michael Poor
Darren Guinness
ABSTRACT
Natural interactions such as speech and gestures have achieved mainstream success independently, with consumer products such as Leap Motion popularizing gestures, while mobile phones have embraced speech input. In this paper we designed an interaction style that combines both gestures and speech to evaluate point and select interaction. Our results indicate that while gestures are slower than the mouse, the introduction of speech allows for selection to be performed without negatively impacting navigation. We also found that users can adapt to this interaction quickly and are able to improve their performance with minimal training. This lays the foundation for future work, such as mouse replacement technologies for those with hand impairments.
- Hauptmann, A. G., and McAvinney, P. Gestures with speech for graphic manipulation. International Journal of Man-Machine Studies 38, 2 (1993), 231--249. Google Scholar
Digital Library
- Jude, A., Poor, G. M., and Guinness, D. Personal space: User defined gesture space for gui interaction. In CHI '14 Extended Abstracts on Human Factors in Computing Systems, CHI EA '14, ACM (New York, NY, USA, 2014), 1615--1620. Google ScholarDigital Library
- Kopp, S. Giving interaction a hand: Deep models of co-speech gesture in multimodal systems. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction, ICMI '13, ACM (New York, NY, USA, 2013), 245--246. Google ScholarDigital Library
- Oviatt, S. User-centered modeling for spoken language and multimodal interfaces. IEEE multimedia 3, 4 (1996), 26--35. Google ScholarDigital Library
- Shneiderman, B. The limits of speech recognition. Commun. ACM 43, 9 (Sept. 2000), 63--65. Google ScholarDigital Library
- Soukoreff, R. W., and MacKenzie, I. S. Towards a standard for pointing device evaluation, perspectives on 27 years of fitts law research in hci. International Journal of Human-Computer Studies 61, 6 (2004), 751--789. Google ScholarDigital Library
Index Terms
- Evaluating multimodal interaction with gestures and speech for point and select tasks
Recommendations
Gestures with speech for hand-impaired persons
ASSETS '14: Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibilityMid-air hand-gestural interaction generally causes a fatigue due to implementations that require the user to hold their arm out during this interaction. Recent research has discovered a new approach to reduce fatigue related to gestural interaction, by ...
An evaluation of touchless hand gestural interaction for pointing tasks with preferred and non-preferred hands
NordiCHI '14: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, FoundationalPerformance evaluations of touchless gestural interaction are generally done by benchmarking pointing performance against existing interactive devices, requiring the use of user's preferred hand. However, as there is no reason for this interaction to be ...
Bimanual Word Gesture Keyboards for Mid-air Gestures
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing SystemsMid-air hand gestural interaction has generally been researched as a pointing device. However, recent research has shown potential for text input with the use of word gesture keyboards (WGK), where these forms of interactions require the input system to ...
Comments