Abstract
Web applications are a widely-spread and a widely-used concept for presenting information. Their underlying architecture and standards, in many cases, limit their presentation/control capabilities of showing pre-recorded audio/video sequences. Highly-dynamic text content, for instance, can only be displayed in its native from (as part of HTML content). This paper provides concepts and answers that enable the transformation of dynamic web-based content into multimodal sequences generated by different multimodal services. Based on the encapsulation of the content into a multimodal shell, any text-based data can dynamically and at interactive speeds be transformed into multimodal visually-synthesized speech. Techniques for the integration of multimodal input (e.g. visioning and speech recognition) are also included. The concept of multimodality relies on mashup approaches rather than traditional integration. It can, therefore, extended any type of web-based solution transparently with no major changes to either the multimodal services or the enhanced web-application.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
EMMA: Extensible MultiModal Annotation Markup Language. W3C Recommendation (2009), http://www.w3.org/TR/2009/REC-emma-20090210/
Hakkinen, M., Dewitt, J.: WebSpeak: user interface design of an accessible web browser. White Paper, the Productivity Works Inc. (1996)
Zajicek, M., Powell, C., Reeves, C.: A web navigation tool for the blind. In: Proceedings of the 3rd ACM/SIGAPH on Assistive Technologies, pp. 204–206 (1998)
Rojc, M., Kačič, Z.: Time and Space-Efficient Architecture for a Corpus-based Text-to-Speech Synthesis System. Speech Communication 49(3), 230–249 (2007)
Yu, W., Kuber, R., Murphy, E., Strain, P., McAllister, G.: A novel multimodal interface for improving visually impaired people’s web accessibility. Virtual Reality 9(2), 133–148 (2006)
Oviatt, S., Cohen, P.: Perceptual user interfaces: multimodal interfaces that process what comes naturally. Communications of the ACM 43(3), 45–53 (2000)
Niklfeld, G., Anegg, H.: Device independent mobile multimodal user interfaces with the MONA Multimodal Presentation Server. In: Proceedings of Eurescom Summit 2005 (2005)
Song, K., Lee, K.H.: Generating multimodal user interfaces for Web services. Interacting with Computers Archive 20(4-5) (September 2008)
Chang, S.E., Minkin, B.: The implementation of a secure and pervasive multimodal Web system architecture. Information and Software Technology 48(6) (2006)
Berti, S., Paternò, F.: Migratory MultiModal Interfaces in MultiDevice Environments. In: Proc. of 7th Int. Conf. on Multimodal Interfaces ICMI 2005. ACM Press, New York (2005)
Bouchet, J., Nigay, L., Ganille, T.: ICARE software components for rapidly developing multimodal interface. In: Conference Proceedings of ICMI 2004 (2004)
Wahlster, W.: SmartWeb: Mobile Applications of the Semantic Web. In: Biundo, S., Frühwirth, T., Palm, G. (eds.) KI 2004. LNCS (LNAI), vol. 3238, pp. 50–51. Springer, Heidelberg (2004)
Stanciulescu, A., Vanderdonckt, J.: Design Options for Multimodal Web Applications. In: Computer Aided Design of User Interfaces V, pp. 41–56 (2007)
Rojc, M., Mlakar, I.: Finite-state machine based distributed framework DATA for intelligent ambience systems. In: Proceedings of CIMMACS 2009, WSEAS Press (2009)
Mlakar, I., Rojc, M.: Platform for flexible integration of multimodal technologies into web application domain. In: Proceedings of E-ACTIVITIES 2009, International Conference on Information Security and Privacy (ISP 2009), WSEAS Press (2009)
Thang, M.D., Dimitrova, V., Djemame, K.: Personalised Mashups Opportunities and Challenges for User Modelling. In: Conati, C., McCoy, K., Paliouras, G. (eds.) UM 2007. LNCS (LNAI), vol. 4511, pp. 415–419. Springer, Heidelberg (2007)
Mlakar, I., Rojc, M.: EVA: expressive multipart virtual agent performing gestures and emotions. International Journal of Mathematics and Computers in Simulation 5(1), 36–44 (2011), http://www.naun.org/journals/mcs/19-710.pdf
Morency, L.P., de Kok, I., Jonathan Gratch, J.: A probabilistic multimodal approach for predicting listener backchannels. Autonomous Agents and Multi-Agent Systems 20(1), 70–84 (2010)
Wrede, B., Kopp, S., Rohlfing, K., Lohse, M., Muhl, C.: Appropriate feedback in asymmetric interactions. Journal of Pragmatics 42(9), 2369–2384 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mlakar, I., Rojc, M. (2011). Developing Multimodal Web Interfaces by Encapsulating Their Content and Functionality within a Multimodal Shell. In: Esposito, A., Vinciarelli, A., Vicsi, K., Pelachaud, C., Nijholt, A. (eds) Analysis of Verbal and Nonverbal Communication and Enactment. The Processing Issues. Lecture Notes in Computer Science, vol 6800. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25775-9_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-25775-9_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-25774-2
Online ISBN: 978-3-642-25775-9
eBook Packages: Computer ScienceComputer Science (R0)