poster

Ambiguity detection in multimodal systems

Authors:

Maria Chiara Caschera,

Fernando Ferri,

Patrizia GrifoniAuthors Info & Claims

AVI '08: Proceedings of the working conference on Advanced visual interfaces

Pages 331 - 334

https://doi.org/10.1145/1385569.1385625

Published: 28 May 2008 Publication History

Abstract

Multimodal systems support users to communicate in a natural way according to their needs. However, the naturalness of the interaction implies that it is hard to find one and only one interpretation of the users' input. Consequently the necessity to define methods for users' input interpretation and ambiguity detection is arising. This paper proposes a theoretical approach based on a Constraint Multiset Grammar combined with Linear Logic, for representing and detecting ambiguities, and in particular semantic ambiguities, produced by the user's input. It considers user's input as a set of primitives defined as terminal elements of the grammar, composing multimodal sentences. The Linear Logic is used to define rules that allow detecting ambiguities connected to the semantics of the user's input. In particular, the paper presents the main features of the user's input and connections between the elements belonging to a multimodal sentence, and it enables to detect ambiguities that can arise during their interpretation process.

References

[1]

J.-Y. Girard. 1987. Linear logic. Theoretical Computer Science, 50. pp. 1--102.

Digital Library

[2]

Sitt Sen Chok, K. Marriott. 1995. "Automatic construction of user interfaces from constraint multiset grammars," vl, 11th International IEEE Symposium on Visual Languages. p. 242--245.

Digital Library

[3]

Johnston, M. and S. Bangalore. 2005. Finite-state Multimodal Integration and Understanding. Journal of Natural Language Engineering 11.2, Cambridge University Press. pp. 159--187.

Digital Library

[4]

Johnston, M. and S. Bangalore. 2005. "Combining Stochastic and Grammar-based Language Processing with Finite-state Edit Machines". In Proceedings of IEEE Automatic Speech Recognition and Understanding Workshop.

[5]

Joyce Yue Chai, Zahar Prasov, Shaolin Qu. 2006. Cognitive Principles in Robust Multimodal Interpretation. J. Artif. Intell. Res. (JAIR) 27: pp. 55--83.

Digital Library

[6]

Tsai, W. H., & Fu, K. S. 1979. Error-correcting isomorphism of attributed relational graphs for pattern analysis. IEEE Trans. Sys., Man and Cyb., 9, pp. 757--768.

[7]

Chai, J. Y., Hong, P., & Zhou, M. X. 2004. A probabilistic approach to reference resolution in multimodal user interfaces. In Proceedings of 9th International Conference on Intelligent User Interfaces (IUI), pp. 70--77.

Digital Library

[8]

M. Collins. 1997. Three generative, lexicalised models for statistical parsing. In Proceedings of the 35th Meeting of the Association for Computational Linguistics and the 7th Conference of the European Chapter of the ACL. pp. 16--23.

Digital Library

[9]

Marsal Gavalda and A. Waibel. 1998. Growing Semantic Grammars. Proceedings of ACL / Coling 1998, Montreal, Canada.

Digital Library

[10]

Carpenter, B. 1992. The Logic of Typed Feature Structures. Cambridge University Press.

Digital Library

[11]

D. M. Berry, E. Kamsties, M. M. Krieger. 2003. From contract drafting to software specification: Linguistic sources of ambiguity. A Handbook. University of Waterloo, Waterloo, Ontario, Canada, 2003.

[12]

K. Marriott, B. Meyer, and K. Wittenburg. A survey of visual language specification and recognition. In K. Marriott and B. Meyer, editors, Visual Language Theory, Springer, New York, 1998. pages 5--85.

Digital Library

[13]

D'Ulizia A, Ferri F., Grifoni P. 2007. A Hybrid Grammar-Based Approach to Multimodal Languages Specification, OTM 2007 Workshop Proceedings, 25--30 November 2007, Vilamoura, Portugal, Springer-Verlag, LNCS 4805. pp 367--376.

Digital Library

[14]

Martin J. C. (1997). Toward Intelligent Cooperation Between Modalities: The Example of a System Enabling Multimodal Interaction with a Map. Proceedings of International Joint Conference on Artificial Intelligence (IJCAI'97) Workshop on "Intelligent Multimodal Systems." Nagoya, Japan.

[15]

Caschera M. C., Ferri F., Grifoni P. 2007. An Approach for Managing 45 Ambiguities in Multimodal Interaction. OTM 2007 Ws, Part I, LNCS 4805. 45 Springer-Verlag Berlin Heidelberg 2007. pp. 387--397.

Digital Library

Cited By

Caschera MGrifoni PFerri F(2022)Emotion Classification from Speech and Text in Videos Using a Multimodal ApproachMultimodal Technologies and Interaction10.3390/mti60400286:4(28)Online publication date: 12-Apr-2022
https://doi.org/10.3390/mti6040028
Grifoni PCaschera MFerri F(2020)Evaluation of a dynamic classification method for multimodal ambiguities based on Hidden Markov ModelsEvolving Systems10.1007/s12530-020-09344-3Online publication date: 23-May-2020
https://doi.org/10.1007/s12530-020-09344-3
Caschera MD’Ulizia AFerri FGrifoni P(2015)Multimodal Systems: An Excursus of the Main Research QuestionsOn the Move to Meaningful Internet Systems: OTM 2015 Workshops10.1007/978-3-319-26138-6_59(546-558)Online publication date: 28-Oct-2015
https://doi.org/10.1007/978-3-319-26138-6_59
Show More Cited By

Index Terms

Ambiguity detection in multimodal systems
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
2. Theory of computation
  1. Formal languages and automata theory
    1. Grammars and context-free languages

Recommendations

An Approach for Managing Ambiguities in Multimodal Interaction
On the Move to Meaningful Internet Systems 2007: OTM 2007 Workshops
Abstract
Multimodal systems support people with different needs and different features during the interaction process making it more easy and natural. However naturalness can usually produce ambiguous interpretation. This paper discusses ambiguities in ...
An approach for managing ambiguities in multimodal interaction
OTM'07: Proceedings of the 2007 OTM confederated international conference on On the move to meaningful internet systems - Volume Part I

Multimodal systems support people with different needs and different features during the interaction process making it more easy and natural. However naturalness can usually produce ambiguous interpretation. This paper discusses ambiguities in ...
Multimodal slideshow: demonstration of the openinterface interaction development environment
ICMI '08: Proceedings of the 10th international conference on Multimodal interfaces

In this paper, we illustrate the OpenInterface Interaction Development Environment (OIDE) that addresses the design and development of multimodal interfaces. Multimodal interaction software development presents a particular challenge because of the ever ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AVI '08: Proceedings of the working conference on Advanced visual interfaces

May 2008

483 pages

ISBN:9781605581415

DOI:10.1145/1385569

General Chair:
Stefano Levialdi
Sapienza, Università di Roma, Roma, Italy

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI Italy: SIGCHI Italy
SIGMM: ACM Special Interest Group on Multimedia
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

AVI '08

Sponsor:

SIGCHI Italy
SIGMM
SIGCHI

AVI '08: The International Conference on Advanced Visual Interfaces

May 28 - 30, 2008

Napoli, Italy

Acceptance Rates

Overall Acceptance Rate 128 of 490 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
162
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Caschera MGrifoni PFerri F(2022)Emotion Classification from Speech and Text in Videos Using a Multimodal ApproachMultimodal Technologies and Interaction10.3390/mti60400286:4(28)Online publication date: 12-Apr-2022
https://doi.org/10.3390/mti6040028
Grifoni PCaschera MFerri F(2020)Evaluation of a dynamic classification method for multimodal ambiguities based on Hidden Markov ModelsEvolving Systems10.1007/s12530-020-09344-3Online publication date: 23-May-2020
https://doi.org/10.1007/s12530-020-09344-3
Caschera MD’Ulizia AFerri FGrifoni P(2015)Multimodal Systems: An Excursus of the Main Research QuestionsOn the Move to Meaningful Internet Systems: OTM 2015 Workshops10.1007/978-3-319-26138-6_59(546-558)Online publication date: 28-Oct-2015
https://doi.org/10.1007/978-3-319-26138-6_59
Caschera MD’Ulizia AFerri FGrifoni P(2014)Multiculturality and Multimodal LanguagesCross-Cultural Interaction10.4018/978-1-4666-4979-8.ch058(1027-1042)Online publication date: 2014
https://doi.org/10.4018/978-1-4666-4979-8.ch058
Caschera MD'Ulizia AFerri FGrifoni P(2014)An Italian Multimodal CorpusProceedings of the Confederated International Workshops on On the Move to Meaningful Internet Systems: OTM 2014 Workshops - Volume 884210.1007/978-3-662-45550-0_57(557-566)Online publication date: 27-Oct-2014
https://dl.acm.org/doi/10.1007/978-3-662-45550-0_57
Caschera MFerri FGrifoni P(2013)InteSe: An Integrated Model for Resolving Ambiguities in Multimodal SentencesIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMCA.2012.221040743:4(911-931)Online publication date: Jul-2013
https://doi.org/10.1109/TSMCA.2012.2210407
Caschera MD’Ulizia AFerri FGrifoni P(2012)Multiculturality and Multimodal LanguagesMultiple Sensorial Media Advances and Applications10.4018/978-1-60960-821-7.ch005(99-114)Online publication date: 2012
https://doi.org/10.4018/978-1-60960-821-7.ch005
Kannan RAndres FFerri FGrifoni P(2011)Towards Multimodal Capture, Annotation and Semantic Retrieval from Performing ArtsAdvances in Computing and Communications10.1007/978-3-642-22726-4_10(79-88)Online publication date: 2011
https://doi.org/10.1007/978-3-642-22726-4_10

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten