skip to main content
10.1145/1385569.1385625acmconferencesArticle/Chapter ViewAbstractPublication PagesaviConference Proceedingsconference-collections
poster

Ambiguity detection in multimodal systems

Published: 28 May 2008 Publication History

Abstract

Multimodal systems support users to communicate in a natural way according to their needs. However, the naturalness of the interaction implies that it is hard to find one and only one interpretation of the users' input. Consequently the necessity to define methods for users' input interpretation and ambiguity detection is arising. This paper proposes a theoretical approach based on a Constraint Multiset Grammar combined with Linear Logic, for representing and detecting ambiguities, and in particular semantic ambiguities, produced by the user's input. It considers user's input as a set of primitives defined as terminal elements of the grammar, composing multimodal sentences. The Linear Logic is used to define rules that allow detecting ambiguities connected to the semantics of the user's input. In particular, the paper presents the main features of the user's input and connections between the elements belonging to a multimodal sentence, and it enables to detect ambiguities that can arise during their interpretation process.

References

[1]
J.-Y. Girard. 1987. Linear logic. Theoretical Computer Science, 50. pp. 1--102.
[2]
Sitt Sen Chok, K. Marriott. 1995. "Automatic construction of user interfaces from constraint multiset grammars," vl, 11th International IEEE Symposium on Visual Languages. p. 242--245.
[3]
Johnston, M. and S. Bangalore. 2005. Finite-state Multimodal Integration and Understanding. Journal of Natural Language Engineering 11.2, Cambridge University Press. pp. 159--187.
[4]
Johnston, M. and S. Bangalore. 2005. "Combining Stochastic and Grammar-based Language Processing with Finite-state Edit Machines". In Proceedings of IEEE Automatic Speech Recognition and Understanding Workshop.
[5]
Joyce Yue Chai, Zahar Prasov, Shaolin Qu. 2006. Cognitive Principles in Robust Multimodal Interpretation. J. Artif. Intell. Res. (JAIR) 27: pp. 55--83.
[6]
Tsai, W. H., & Fu, K. S. 1979. Error-correcting isomorphism of attributed relational graphs for pattern analysis. IEEE Trans. Sys., Man and Cyb., 9, pp. 757--768.
[7]
Chai, J. Y., Hong, P., & Zhou, M. X. 2004. A probabilistic approach to reference resolution in multimodal user interfaces. In Proceedings of 9th International Conference on Intelligent User Interfaces (IUI), pp. 70--77.
[8]
M. Collins. 1997. Three generative, lexicalised models for statistical parsing. In Proceedings of the 35th Meeting of the Association for Computational Linguistics and the 7th Conference of the European Chapter of the ACL. pp. 16--23.
[9]
Marsal Gavalda and A. Waibel. 1998. Growing Semantic Grammars. Proceedings of ACL / Coling 1998, Montreal, Canada.
[10]
Carpenter, B. 1992. The Logic of Typed Feature Structures. Cambridge University Press.
[11]
D. M. Berry, E. Kamsties, M. M. Krieger. 2003. From contract drafting to software specification: Linguistic sources of ambiguity. A Handbook. University of Waterloo, Waterloo, Ontario, Canada, 2003.
[12]
K. Marriott, B. Meyer, and K. Wittenburg. A survey of visual language specification and recognition. In K. Marriott and B. Meyer, editors, Visual Language Theory, Springer, New York, 1998. pages 5--85.
[13]
D'Ulizia A, Ferri F., Grifoni P. 2007. A Hybrid Grammar-Based Approach to Multimodal Languages Specification, OTM 2007 Workshop Proceedings, 25--30 November 2007, Vilamoura, Portugal, Springer-Verlag, LNCS 4805. pp 367--376.
[14]
Martin J. C. (1997). Toward Intelligent Cooperation Between Modalities: The Example of a System Enabling Multimodal Interaction with a Map. Proceedings of International Joint Conference on Artificial Intelligence (IJCAI'97) Workshop on "Intelligent Multimodal Systems." Nagoya, Japan.
[15]
Caschera M. C., Ferri F., Grifoni P. 2007. An Approach for Managing 45 Ambiguities in Multimodal Interaction. OTM 2007 Ws, Part I, LNCS 4805. 45 Springer-Verlag Berlin Heidelberg 2007. pp. 387--397.

Cited By

View all
  • (2022)Emotion Classification from Speech and Text in Videos Using a Multimodal ApproachMultimodal Technologies and Interaction10.3390/mti60400286:4(28)Online publication date: 12-Apr-2022
  • (2020)Evaluation of a dynamic classification method for multimodal ambiguities based on Hidden Markov ModelsEvolving Systems10.1007/s12530-020-09344-3Online publication date: 23-May-2020
  • (2015)Multimodal Systems: An Excursus of the Main Research QuestionsOn the Move to Meaningful Internet Systems: OTM 2015 Workshops10.1007/978-3-319-26138-6_59(546-558)Online publication date: 28-Oct-2015
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
AVI '08: Proceedings of the working conference on Advanced visual interfaces
May 2008
483 pages
ISBN:9781605581415
DOI:10.1145/1385569
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2008

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. grammar-based language
  2. interpretation of multimodal input
  3. multimodal ambiguity
  4. multimodal interfaces

Qualifiers

  • Poster

Conference

AVI '08
Sponsor:

Acceptance Rates

Overall Acceptance Rate 128 of 490 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)0
Reflects downloads up to 25 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Emotion Classification from Speech and Text in Videos Using a Multimodal ApproachMultimodal Technologies and Interaction10.3390/mti60400286:4(28)Online publication date: 12-Apr-2022
  • (2020)Evaluation of a dynamic classification method for multimodal ambiguities based on Hidden Markov ModelsEvolving Systems10.1007/s12530-020-09344-3Online publication date: 23-May-2020
  • (2015)Multimodal Systems: An Excursus of the Main Research QuestionsOn the Move to Meaningful Internet Systems: OTM 2015 Workshops10.1007/978-3-319-26138-6_59(546-558)Online publication date: 28-Oct-2015
  • (2014)Multiculturality and Multimodal LanguagesCross-Cultural Interaction10.4018/978-1-4666-4979-8.ch058(1027-1042)Online publication date: 2014
  • (2014)An Italian Multimodal CorpusProceedings of the Confederated International Workshops on On the Move to Meaningful Internet Systems: OTM 2014 Workshops - Volume 884210.1007/978-3-662-45550-0_57(557-566)Online publication date: 27-Oct-2014
  • (2013)InteSe: An Integrated Model for Resolving Ambiguities in Multimodal SentencesIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMCA.2012.221040743:4(911-931)Online publication date: Jul-2013
  • (2012)Multiculturality and Multimodal LanguagesMultiple Sensorial Media Advances and Applications10.4018/978-1-60960-821-7.ch005(99-114)Online publication date: 2012
  • (2011)Towards Multimodal Capture, Annotation and Semantic Retrieval from Performing ArtsAdvances in Computing and Communications10.1007/978-3-642-22726-4_10(79-88)Online publication date: 2011

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media