Adam Faeth
ABSTRACT
The simple action of pressing a button is a multimodal interaction with an interesting depth of complexity. As the development of computer interfaces supporting 3D tasks progresses, there is a need to understand how users will interact with virtual buttons that generate multimodal feedback. Using a phone number dialing task on a virtual keypad, this study examined the effects of visual, auditory, and haptic feedback combinations on task performance and on the motion of individual button presses. The results suggest that the resistance of haptic feedback alone was not enough to prevent participants from pressing the button farther than necessary. Reinforcing haptic feedback with visual or auditory feedback shortened the depth of the presses significantly. However, the shallower presses that occurred with trimodal feedback may have led participants to release some buttons too early, which may explain an unexpected increase in mistakes where the participant missed digits from the phone number.
- E. M. Altinsoy. Perceptual aspects of auditory-tactile asynchrony. In Proc. ICSV 10, pages 3831--3838, 2003.Google Scholar
- J. L. Burke, M. S. Prewett, A. A. Gray, L. Yang, F. R. B. Stilson, M. D. Coovert, L. R. Elliot, and E. Redden. Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis. In Proc. ICMI '06, pages 108--117. ACM, Nov. 2006. Google ScholarDigital Library
- C. Clare. Human factors: A most important ingredient in keyboard designs. EDN Magazine, 21(8):99--102, 1976.Google Scholar
- K. M. Cohen. Membrane keyboards and human performance. Human Factors and Ergonomics Society Annual Meeting Proceedings, 26:424--424, 1982.Google ScholarCross Ref
- F. Colavita. Human sensory dominance. Perception and Psycophysics, 16(2):409--412, 1974.Google ScholarCross Ref
- I. n. Díaz, J. Hernantes, I. Mansa, A. Lozano, D. Borro, J. J. Gil, and E. Sánchez. Influence of multisensory feedback on haptic accessibility tasks. Virtual Reality, 10(1):31--40, May 2006.Google ScholarDigital Library
- D. E. DiFranco, G. L. Beauregard, and M. A. Srinivasan. The effect of auditory cues on the haptic perception of stiffness in virtual environments. In ASME Dynamic Systems and Control Division, volume 61, pages 17--22. ASME, 1997.Google Scholar
- Y. Guiard. Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior, 19:486--517, 1987.Google ScholarCross Ref
- D. Hecht and M. Reiner. Sensory dominance in combinations of audio, visual and haptic stimuli. Experimental brain research, 193:307--314, Jan 2009.Google Scholar
- D. Hecht, M. Reiner, and A. Karni. Enhancement of response times to bi- and tri-modal sensory stimuli during active movements. Experimental brain research, 185(4):655--665, Mar. 2008.Google Scholar
- M. A. Heller, J. A. Calcaterra, S. L. Green, and L. Brown. Intersensory conflict between vision and touch: the response modality dominates when precise, attention-riveting judgments are required. Perception & psychophysics, 61(7):1384--1398, Oct. 1999.Google ScholarCross Ref
- M. Hershenson. Reaction time as a measure of intersensory facilitation. Journal of experimental psychology, 63:289--293, Mar. 1962.Google Scholar
- E. Hoggan, T. Kaaresoja, P. Laitinen, and S. Brewster. Crossmodal congruence: the look, feel and sound of touchscreen widgets. Proc. IMCI '08, pages 157--164, Oct. 2008. Google ScholarDigital Library
- R. Kinkead and B. Gonzalez. Human factors design recommendations for touch-operated keyboards -- final report. Technical Report 12091-FR, Honeywell, Inc., March 1969.Google Scholar
- R. Klatzky and S. J. Lederman. Touch. In I. B. Weiner, editor, Handbook of Psychology, pages 147--176. Wiley, Hoboken, NJ, USA, 2003.Google Scholar
- R. Klatzky and S. J. Lederman. Object Recognition by Touch. In Blindness and Brain Plasticity in Navigation and Object Perception, pages 185--207. 2008.Google Scholar
- A. Kohlrausch and S. van de Par. Auditory-visual interaction: from fundamental research in cognitive psychology to (possible) applications. In Proc. SPIE, pages 34--44, 1999.Google Scholar
- A. Lécuyer, J.-M. Burkhardt, S. Coquillart, and P. Coiffet. "boundary of illusion:" an experiment of sensory integration with a pseudo-haptic system. In Proc. VR'01, VR '01, pages 115--122, Washington, DC, USA, 2001. IEEE Computer Society. Google ScholarDigital Library
- S. Lee and S. Zhai. The performance of touch screen soft buttons. In ACM CHI '09, pages 309--318, 2009. Google ScholarDigital Library
- V. Levenshtein. Binary codes capable of correcting deletions, insertions, and reversals. Problems in Information Transmission, pages 8--17, 1965.Google Scholar
- J. R. Lewis, K. M. Potosnak, and R. L. Magyar. Keys and Keyboards, chapter 54, pages 1285--1315. Elsevier, Amsterdam, 1997.Google Scholar
- J. Long. Effects of delayed irregular feedback on unskilled and skilled keying performance. Ergonomics, 19(2):183--202, 1976.Google ScholarCross Ref
- T. H. Massie and J. K. Salisbury. The phantom haptic interface: A device for probing virtual objects. In ASME Winter Annual Meeting, 1994.Google Scholar
- H. McGurk and J. MacDonald. Hearing lips and seeing voices. Nature, 264(5588):746--748, Dec. 1976.Google ScholarCross Ref
- Microsoft. QueryPerformanceCounter. http://msdn.microsoft.com/en-us/library/windows/desktop/ms644904, March 2012.Google Scholar
- N. Miner, B. Gillespie, and T. Caudell. Examining the influence of audio and visual stimuli on a haptic display. In Proc. IMAGE '96, 1996.Google Scholar
- A. Nashel and S. Razzaque. Tactile virtual buttons for mobile devices. In ACM CHI '03, pages 854--855, 2003. Google ScholarDigital Library
- S. Oviatt. Multimodal Interfaces, chapter 21, pages 413--432. CRC Press, 2007.Google Scholar
- D. Pollard and M. Cooper. The effect of feedback on keying performance. Applied Ergonomics, 10(4):194--200, 1979.Google ScholarCross Ref
- C. J. Roe, W. H. Muto, and T. Blake. Feedback and key discrimination on membrane keypads. Human Factors and Ergonomics Society Annual Meeting, 28:277--281, 1984.Google ScholarCross Ref
- C. Spence, F. Pavani, and J. Driver. Crossmodal links between vision and touch in covert endogenous spatial attention. J. of Experimental Psychology: Human Perception and Performance, 26(4):1298--1319, 2000.Google ScholarCross Ref
- M. A. Srinivasan, G. L. Beauregard, and D. L. Brock. The impact of visual information on the haptic perception of stiffness in virtual environments. In ASME Winter Annual Meeting, volume 58, pages 555--559, 1996.Google Scholar
- H. S. Vitense, J. A. Jacko, and V. K. Emery. Multimodal feedback: an assessment of performance and mental workload. Ergonomics, 46(1--3):68--87, 2003.Google Scholar
- R. A. Wagner and M. J. Fischer. The String-to-String Correction Problem. Journal of the ACM, 21(1):168--173, 1974. Google ScholarDigital Library
- P. Walker and S. Smith. Stroop interference based on the multimodal correlates of haptic size and auditory pitch. Perception, 14(6):729--736, 1985.Google ScholarCross Ref
- C. D. Wickens. Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3(2):159--177, 2002.Google ScholarCross Ref
Index Terms
- Effects of modality on virtual button motion and performance
Recommendations
Emergent effects in multimodal feedback from virtual buttons
The continued advancement in computer interfaces to support 3D tasks requires a better understanding of how users will interact with 3D user interfaces in a virtual workspace. This article presents two studies that investigated the effect of visual, ...
Virtual Buttons for Eyes-Free Interaction: A Study
Human-Computer Interaction – INTERACT 2015AbstractThe touch screen of mobile devices, such as smart phones and tablets, is their primary input mechanism. While designed to be used in conjunction with its output capabilities, eyes-free interaction is also possible and useful on touch screens. ...
Feel-good touch: finding the most pleasant tactile feedback for a mobile touch screen button
ICMI '08: Proceedings of the 10th international conference on Multimodal interfacesEarlier research has shown the benefits of tactile feedback for touch screen widgets in all metrics: performance, usability and user experience. In our current research the goal was to go deeper in understanding the characteristics of a tactile click ...
Comments