Skip to main content
Springer Nature Link
Log in

Formalization of Multimodal Languages in Pervasive Computing Paradigm

  • Conference paper
Advanced Internet Based Systems and Applications (SITIS 2006)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 4879))

  • 384 Accesses

Abstract

The pervasive computing paradigm provides the user with a uniform computing space available everywhere, any time and in the most appropriate form and modality. These aspects produce the need for user interfaces that are usable, multimodal and personalized for each user. In this paper multimodality is discussed. In particular, features and computational issues of the multimodal interaction are analyzed in order to examine methodological aspects for the definition of multimodal interaction languages for pervasive applications. The multimodality is faced at a grammar level, rather than at a dialogue management level. This means that different unimodal inputs are considered as a unique multimodal input that is sent to the dialogue parser that uses the grammar specification to interpret it, rather than to be distinctly interpreted and then combined. The main objective of the paper is thus to explore multimodal interaction for pervasive applications through the use of a multimodal language rather than through the integration of several unimodal languages.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Coutaz, J., Caelen, J.: A Taxonomy For Multimedia and Multimodal User Interfaces. In: Proceedings of the 1st ERCIM Workshop on Multimedia HCI, Lisbon (November 1991)

    Google Scholar 

  2. Mueller, W., Schaefer, R., Bleul, S.: Interactive Multimodal User Interfaces for Mobile Devices. In: 37th Hawaii International Conference on System Sciences (HICSS 2004) (2004)

    Google Scholar 

  3. Bouchet, J., Nigay, L.: Balzagette, D., ICARE: Approche à composants pour l’interaction multimodale. In: Actes des Premières Journées Francophones: Mobilité et Ubiquité 2004, Nice Sophia-Antipolis, France, pp. 36–43 (June 2004)

    Google Scholar 

  4. Simon, R., Wegscheider, F., Tolar, K.: Tool-Supported Single Authoring for Device Independence and Multimodality. In: 7th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2005) (2005)

    Google Scholar 

  5. Carbonell, N.: Towards the design of usable multimodal interaction languages. Journal on Universal Access in the Information Society (UAIS 2003) 2(2), 143–159 (2003)

    Article  Google Scholar 

  6. G"ottler, H.: Graph grammars, a new paradigm for implementing visual languages. In: Eurographics 1989, pp. 505–516. ElsevierScience publishers, North-Holland (1986)

    Google Scholar 

  7. Göttler, H.: Graph grammars and diagram editing. In: Ehrig, H., Nagl, M., Rosenfeld, A., Rozenberg, G. (eds.) Graph Grammars 1986. LNCS, vol. 291, pp. 211–231. Springer, Heidelberg (1987)

    Chapter  Google Scholar 

  8. Crimi, C., Guercio, A., Nota, G., Pacini, G., Tortora, G., Tucci, M.: Relation grammars for modelling multi-dimensional structures. In: IEEE Symposium on Visual Languages, pp. 168–173. IEEE Computer Society Press, Los Alamitos (1990)

    Google Scholar 

  9. Helm, R., Marriott, K., Odersky, M.: Building visual language parsers. In: ACM Conf. Human Factors in Computing, pp. 118–125 (1991)

    Google Scholar 

  10. Marriott, K.: Constraint multiset grammars. In: IEEE Symposium on Visual Languages, pp. 118–125. IEEE Computer Society Press, Los Alamitos (1994)

    Google Scholar 

  11. Marriott, K., Meyer, B., Wittenburg, K.: A survey of visual language specification and recognition. In: Marriott, K., Meyer, B. (eds.) Visual Language Theory, pp. 5–85. Springer, New York (1998)

    Chapter  Google Scholar 

  12. Stivers, T., Sidnell, J.: Introduction: Multimodal interaction. Semiotica 156(1/4), 1–20 (2005)

    Article  Google Scholar 

  13. Thrisson, K.R.: Computational Characteristics of Multimodal Dialogue. In: AAAI Fall Symposium on Embodied Language and Action, Massachusetts Institute of Technology, Cambridge, Massachusetts, November 10-12, pp. 102–108 (1995)

    Google Scholar 

  14. Gupta, A.: An adaptive approach to collecting multimodal input. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (2003)

    Google Scholar 

  15. Oviatt, S., DeAngeli, A., Kuhn, K.: Integration and Synchronization of Input Modes during Multimodal Human-Computer Interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 415–422 (1997)

    Google Scholar 

  16. Crimi, C., Guercio, A., Nota, G., Pacini, G., Tortora, G., Tucci, M.: Relation grammars for modelling multi-dimensional structures. In: IEEE Symposium on Visual Languages, pp. 168–173. IEEE Computer Society Press, Los Alamitos (1990)

    Google Scholar 

  17. Helm, R., Marriott, K., Odersky, M.: Building visual language parsers. In: ACM Conf. Human Factors in Computing, pp. 118–125 (1991)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IRPPS-CNR, via Nizza 128, 00198, Rome, Italy

    Arianna D’Ulizia & Fernando Ferri

Authors
  1. Arianna D’Ulizia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando Ferri
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartemento Tecnologie dell’Informazione, Universitá degli Studi di Milano, Via Bramante 65, 26013, Crema, Italy

    Ernesto Damiani

  2. LE2I-CNRS, Université de Bourgogne, Aile de l’Ingénieur, 21078, Dijon Cedex, France

    Kokou Yetongnon , Richard Chbeir  & Albert Dipanda ,  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

D’Ulizia, A., Ferri, F. (2009). Formalization of Multimodal Languages in Pervasive Computing Paradigm. In: Damiani, E., Yetongnon, K., Chbeir, R., Dipanda, A. (eds) Advanced Internet Based Systems and Applications. SITIS 2006. Lecture Notes in Computer Science, vol 4879. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01350-8_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-01350-8_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-01349-2

  • Online ISBN: 978-3-642-01350-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics