research-article

Exploration of multimodal input interaction based on goals

Authors:

Anirudha JoshiAuthors Info & Claims

APCHI '13: Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction

Pages 83 - 92

https://doi.org/10.1145/2525194.2525213

Published: 24 September 2013 Publication History

Abstract

Today applications demand newer and newer ways through which humans can interact with the system for effective and natural interaction. Each interaction modality have unique features and supports different interaction goals, which makes them logically appropriate for specific types of interaction necessary for different applications. With several multimodal input technologies now being available for users, a challenge is to design optimal combinations of multiple modalities for specific tasks. Also, it is important to understand that, how these modalities may coherently be used together in a well-coordinated manner. This paper presents our initial research wherein we did exploration with the objective of designing a scheme for identification of optimal multimodal input interaction for various applications and use cases. We identified the key interaction goals of few sample applications or usage scenarios, and systematically compared those with the capabilities of the modalities, in order to identify optimal multimodal combination for those applications.

References

[1]

Thorisson, K. R., Koons, D. B., and Bolt, R. A. Multimodal natural dialogue. In Proc. of the SIGCHI conference on Human factors in computing systems, ACM Press (1992), 653--654.

Digital Library

[2]

Bernsen, N. O., and Dybkjær, L. Evaluation of spoken multimodal conversation. In Proc. of the 6th international conference on Multimodal interfaces, ACM Press (2004), 38--45.

Digital Library

[3]

Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J., and Young, R. Four easy pieces for assessing the usability of multimodal interaction: the CARE properties. In Proc. of INTERACT, (1995), vol. 95, 115--120.

[4]

Wechsung, I., Engelbrecht, K. P., Kühnel, C., Möller, S., and Weiss, B. Measuring the Quality of Service and Quality of Experience of multimodal human--machine interaction. Journal on Multimodal User Interfaces, (2012), 1--13.

[5]

Martin, J. C. TYCOON: Theoretical framework and software tools for multimodal interfaces. Intelligence and Multimodality in Multimedia interfaces, (1998).

[6]

Vernier, F., and Nigay, L. A framework for the combination and characterization of output modalities. In Interactive Systems Design, Specification, and Verification, Springer Berlin Heidelberg, (2001), 35--50.

Digital Library

[7]

Beelders, T. R., and Blignaut, P. J. The Usability of Speech and Eye Gaze as a Multimodal Interface for a Word Processor. Speech Technologies, (2011), 386--404.

[8]

Ren, X., Zhang, G., and Dai, G. An experimental study of input modes for multimodal human-computer interaction. Advances in Multimodal Interfaces---ICMI 2000, ACM Press (2000), 49--56.

Digital Library

[9]

Metze, F., Wechsung, I., Schaffer, S., Seebode, J., and Moller, S. Reliable evaluation of multimodal dialogue systems. Human-Computer Interaction. Novel Interaction Methods and Techniques, (2009), 75--83.

Digital Library

[10]

Bernhaupt, R., Navarre, D., Palanque, P., and Winckler, M. Model-based evaluation: A new way to support usability evaluation of multimodal interactive applications. Maturing Usability: Quality in Software, Interaction and Quality, (2007), 96--122.

[11]

Dybkaer, L., Bernsen, N. O., and Minker, W. New challenges in usability evaluation-beyond task-oriented spoken dialogue systems. In Proc. of ICSLP, (2004), Vol. 3, 2261--2264.

[12]

Naumann, A., Wechsung, I., and Moller, S. Factors Influencing Modality Choice in Multimodal Applications. In Perception in Multimodal Dialogue Systems, (2008), 37--43.

Digital Library

[13]

Oviatt, S. L., and Olsen, E. Integration Themes in Multimodal Human-Computer Interaction. In International Conference on Spoken Language Processing, (1994), vol. 2, 551--554.

[14]

Ghosh, S., Joshi, A., and Tripathi, S. Empirical Evaluation of Multimodal Input Interactions. In Human Interface and the Management of Information. Information and Interaction Design, Springer Berlin Heidelberg, (2013), 37--47.

Digital Library

[15]

International Organization for Standardization, ISO/IEC 9126-1Software Engineering - Product Quality, (2001).

[16]

International Organization for Standardization, ISO 9241-1Ergonomic requirements for office work with visual display terminals (VDTs), (1997).

[17]

Shneiderman B. Designing the User Interface: Strategies for Effective Human-Computer Interaction, Addison Wesley (2004), 4th Ed.

Digital Library

[18]

Nielsen J. Usability Engineering, Morgan Kaufmann, (1993).

Digital Library

[19]

Joshi A., and Sarda N. L. Do Teams Achieve Usability Goals? Evaluating Goal Achievement with Usability Goals Setting Tool, In Proc. of IFIP TC13 International Conference on Human-Computer Interaction INTERACT, (2011).

Digital Library

[20]

Seffah, A., Donyaee M., Kline, R. B., and Padda, H. K. Usability measurement and metrics: A consolidated model. Software Quality Journal, (2006), Vol. 14.2, 159--178.

Digital Library

Cited By

El Kamali MAngelini LLalanne DAbou Khaled OMugellini E(2023)Older adults' perspectives on multimodal interaction with a conversational virtual coachFrontiers in Computer Science10.3389/fcomp.2023.11258955Online publication date: 7-Nov-2023
https://doi.org/10.3389/fcomp.2023.1125895
Lazaro MKim SLee JChun JYun M(2021)Interaction Modalities for Notification Signals in Augmented RealityProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479898(470-477)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479898
(2017)ReferencesUniversal UX Design10.1016/B978-0-12-802407-2.00019-8(215-225)Online publication date: 2017
https://doi.org/10.1016/B978-0-12-802407-2.00019-8
Show More Cited By

Index Terms

Exploration of multimodal input interaction based on goals
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Exploration of Usage Patterns for Multimodal Input Interactions
IndiaHCI '14: Proceedings of the 6th Indian Conference on Human-Computer Interaction

This work attempts to study the usage patterns for multimodal input interactions like, pattern of errors, user's modality preferences when dealing with errors, their pattern of switching across modalities and their preferences of input modalities. In ...
Empirical evaluation of multimodal input interactions
HCI International'13: Proceedings of the 15th international conference on Human Interface and the Management of Information: information and interaction design - Volume Part I

With variety of interaction technologies like speech, pen, touch, hand or body gestures, eye gaze, etc., being now available for users, it is a challenge to design optimal and effective multimodal combinations for specific tasks. For designing that, it ...
Prediction of Various Backchannel Utterances Based on Multimodal Information
IVA '23: Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents

The listener's backchannels are an important part of dialogues. With appropriate backchannels, people are able to smoothly promote dialogues. Thus, backchannels are considered to be important in dialogues between not only humans but also humans and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

APCHI '13: Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction

September 2013

420 pages

ISBN:9781450322539

DOI:10.1145/2525194

General Chairs:
Sanjay Tripathi
ABB Corporate Research, India
,
Anirudha Joshi
IIT Bombay, India

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 September 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

APCHI '13

Sponsor:

SIGCHI

APCHI '13: 11th Asia Pacific Conference on Computer Human Interaction

September 24 - 27, 2013

Bangalore, India

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
220
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

El Kamali MAngelini LLalanne DAbou Khaled OMugellini E(2023)Older adults' perspectives on multimodal interaction with a conversational virtual coachFrontiers in Computer Science10.3389/fcomp.2023.11258955Online publication date: 7-Nov-2023
https://doi.org/10.3389/fcomp.2023.1125895
Lazaro MKim SLee JChun JYun M(2021)Interaction Modalities for Notification Signals in Augmented RealityProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479898(470-477)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479898
(2017)ReferencesUniversal UX Design10.1016/B978-0-12-802407-2.00019-8(215-225)Online publication date: 2017
https://doi.org/10.1016/B978-0-12-802407-2.00019-8
Himmelsbach JGarschall MEgger SSteffek STscheligi MHolzmann CMayrhofer RHäkkilä JRukzio ERoland M(2015)Enabling accessibility through multimodality?Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia10.1145/2836041.2836060(195-199)Online publication date: 30-Nov-2015
https://dl.acm.org/doi/10.1145/2836041.2836060

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten