Abstract
The pervasive computing paradigm provides the user with a uniform computing space available everywhere, any time and in the most appropriate form and modality. These aspects produce the need for user interfaces that are usable, multimodal and personalized for each user. In this paper multimodality is discussed. In particular, features and computational issues of the multimodal interaction are analyzed in order to examine methodological aspects for the definition of multimodal interaction languages for pervasive applications. The multimodality is faced at a grammar level, rather than at a dialogue management level. This means that different unimodal inputs are considered as a unique multimodal input that is sent to the dialogue parser that uses the grammar specification to interpret it, rather than to be distinctly interpreted and then combined. The main objective of the paper is thus to explore multimodal interaction for pervasive applications through the use of a multimodal language rather than through the integration of several unimodal languages.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Coutaz, J., Caelen, J.: A Taxonomy For Multimedia and Multimodal User Interfaces. In: Proceedings of the 1st ERCIM Workshop on Multimedia HCI, Lisbon (November 1991)
Mueller, W., Schaefer, R., Bleul, S.: Interactive Multimodal User Interfaces for Mobile Devices. In: 37th Hawaii International Conference on System Sciences (HICSS 2004) (2004)
Bouchet, J., Nigay, L.: Balzagette, D., ICARE: Approche à composants pour l’interaction multimodale. In: Actes des Premières Journées Francophones: Mobilité et Ubiquité 2004, Nice Sophia-Antipolis, France, pp. 36–43 (June 2004)
Simon, R., Wegscheider, F., Tolar, K.: Tool-Supported Single Authoring for Device Independence and Multimodality. In: 7th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2005) (2005)
Carbonell, N.: Towards the design of usable multimodal interaction languages. Journal on Universal Access in the Information Society (UAIS 2003) 2(2), 143–159 (2003)
G"ottler, H.: Graph grammars, a new paradigm for implementing visual languages. In: Eurographics 1989, pp. 505–516. ElsevierScience publishers, North-Holland (1986)
Göttler, H.: Graph grammars and diagram editing. In: Ehrig, H., Nagl, M., Rosenfeld, A., Rozenberg, G. (eds.) Graph Grammars 1986. LNCS, vol. 291, pp. 211–231. Springer, Heidelberg (1987)
Crimi, C., Guercio, A., Nota, G., Pacini, G., Tortora, G., Tucci, M.: Relation grammars for modelling multi-dimensional structures. In: IEEE Symposium on Visual Languages, pp. 168–173. IEEE Computer Society Press, Los Alamitos (1990)
Helm, R., Marriott, K., Odersky, M.: Building visual language parsers. In: ACM Conf. Human Factors in Computing, pp. 118–125 (1991)
Marriott, K.: Constraint multiset grammars. In: IEEE Symposium on Visual Languages, pp. 118–125. IEEE Computer Society Press, Los Alamitos (1994)
Marriott, K., Meyer, B., Wittenburg, K.: A survey of visual language specification and recognition. In: Marriott, K., Meyer, B. (eds.) Visual Language Theory, pp. 5–85. Springer, New York (1998)
Stivers, T., Sidnell, J.: Introduction: Multimodal interaction. Semiotica 156(1/4), 1–20 (2005)
Thrisson, K.R.: Computational Characteristics of Multimodal Dialogue. In: AAAI Fall Symposium on Embodied Language and Action, Massachusetts Institute of Technology, Cambridge, Massachusetts, November 10-12, pp. 102–108 (1995)
Gupta, A.: An adaptive approach to collecting multimodal input. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (2003)
Oviatt, S., DeAngeli, A., Kuhn, K.: Integration and Synchronization of Input Modes during Multimodal Human-Computer Interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 415–422 (1997)
Crimi, C., Guercio, A., Nota, G., Pacini, G., Tortora, G., Tucci, M.: Relation grammars for modelling multi-dimensional structures. In: IEEE Symposium on Visual Languages, pp. 168–173. IEEE Computer Society Press, Los Alamitos (1990)
Helm, R., Marriott, K., Odersky, M.: Building visual language parsers. In: ACM Conf. Human Factors in Computing, pp. 118–125 (1991)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
D’Ulizia, A., Ferri, F. (2009). Formalization of Multimodal Languages in Pervasive Computing Paradigm. In: Damiani, E., Yetongnon, K., Chbeir, R., Dipanda, A. (eds) Advanced Internet Based Systems and Applications. SITIS 2006. Lecture Notes in Computer Science, vol 4879. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01350-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-01350-8_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-01349-2
Online ISBN: 978-3-642-01350-8
eBook Packages: Computer ScienceComputer Science (R0)