Abstract
Despite optimistic expectations, the spread of multimodal mobile applications is proceeding slowly. Nevertheless the power of new high-end devices gives the opportunity to create a new class of application with advanced synergic multimodal features. In this paper we present the results the CHAT group achieved in defining and building a platform for developing synergic mobile multimodal services. CHAT is a project co-funded by Italian Ministry of Research, aimed at providing multimodal context-sensitive services to mobile users. Our architecture is based on the following key concepts: thin client approach, modular client interface, asynchronous content push, distributed recognition, natural language processing, speech driven semantic fusion. The core of the system is based on a mix of web and telecommunication technologies. This choice proved to be very useful to create high personalized context sensitive services. One of the main features is the possibility to push appropriate contents on the user terminal reducing unfriendly user interactions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Frattini, G., Romano, L., di Carlo, V.S., Petriccione, P., Supino, G., Leone, G., Autiero, C.: Multimodal Architectures: Issues and Experiences. In: Meersman, R., Tari, Z., Herrero, P. (eds.) OTM 2006 Workshops. LNCS, vol. 4277, pp. 974–983. Springer, Heidelberg (2006)
Frattini, G., Gaudino, F., di Carlo, V.S.: Mobile multimodal applications on mass-market devices: experiences - 18th International Workshop on Database and Expert Systems Applications (DEXA 2007) Dexa workshop 2007, pp. 89–93 (2007)
X+V for the Next Generation Web, http://www.voicexml.org/specs/multimodal/
Filippo, F., Krebs, A., Marsic, I.I.: A framework for rapid development of multimodal interfaces. In: Proceedings of the 5th international conference on Multimodal interfaces, Vancouver, British Columbia, Canada, November 05-07 (2003)
Synchronized Multimedia Integration Language (SMIL), http://www.w3.org/TR/REC-smil/
Niklfeld, G., Anegg, H., Pucher, M., Schatz, R., Simon, R., Wegscheider, F., Gassner, A., Jank, M., Pospischil, G.: Device independent mobile multimodal user interfaces with the MONA Multimodal Presentation Server. In: Proceedings of Eurescom Summit 2005, Heidelberg, Germany, April 27-29 (2005)
Thinlet classic (last accessed March 20, 2007), http://www.thinlet.com/
Reitter, D., Panttaja, E., Cummins, F.: UI on the fly: Generating a multimodal user interface. In: Proceedings of Human Language Technology conf 2004 / North American chapter of the Association for Computational Linguistics (HLT/NAACL 2004) (2004)
Piccolo (last accessed February 20, 2008), http://www.cs.umd.edu/hcil/jazz/index.shtml
Loquendo vocal technology and services, http://www.loquendo.com
Jarnal (last accessed March 5, 2008), http://www.dklevine.co/general/software/tc1000/jarnal.htm
Avola, D., Ferri, F., Grifoni, P.: Formalizing Recognition of Sketching Styles in Human Centered Systems. In: Apolloni, B., Howlett, R.J., Jain, L. (eds.) KES 2007, Part II. LNCS (LNAI), vol. 4693, pp. 369–376. Springer, Heidelberg (2007)
Avola, D., Caschera, M.C., Ferri, F., Grifoni, P.: Ambiguities in Sketch-Based Interfaces. In: Proceedings of the Hawaii International Conference on System Sciences (HICSS 2007), p. 290. IEEE Computer Society, Los Alamitos (2007)
Gorin, A.L., Alonso, T., Abella, A., Riccardi, G., Wright, J.H.: Semantic Information Processing of Spoken Language - How May I Help You? In: Pattern recognition in Speech and Language Processing. CRC Press, Boca Raton (2003)
Wright, J.H., Gorin, A.L., Riccardi, G.: Automatic Acquisition of Salient Grammar Fragments for Call-Type Classification. In: Proc. 5th Europ. Conf. Speech Communication and Technology, pp. 1419–1422. Intern. Speech Communication Ass., Bonn, Germany (1997)
Arai, K., Wright, J.H., Riccardi, G., Gorin, A.: Grammar fragment acquisition using syntactic and semantic clustering. Speech Communication 27 (January 1999)
Gildea, D., Jurasfky, D.: Automatic labeling of semantic roles. Computational Linguistic 28(3), 496–530 (2002)
Ng, H.T., Zelle, J.M.: Corpus-Based Approaches to Semantic Interpretation in NLP. AI Magazine 18(4), 45–64 (1997)
Johnson, C.R., Fillmore, C.J.: The FrameNet tagset for frame-semantic and syntactic coding of predicate-argument structure. In: Proceedings of the 1st Meeting of the North American Chapter of the Association for Computational Linguistics (ANLP-NAACL 2000), April 29-May 4, 2000, pp. 56–62 (2000)
W3C multimodal interaction activity, http://www.w3.org/2002/mmi
XUL project (last accessed November 12, 2007), http://www.mozilla.org/projects/xul/
Minker, W., Bennacef, S.: Speech and Human-Machine Dialog. Kluwer Academic Publishers, Boston (2004)
Koons, D.B., Sparrell, C.J., Thorisson, K.R.: Integrating simultaneous input from speech, gaze and hand gestures. In: Maybury, M. (ed.) Intelligent Multimedia Interfaces, Menlo Park, CA, pp. 257–276. MIT, Cambridge (1993)
Russ, G., Sallans, B., Hareter, H.: Semantic Based Information Fusion in a Multimodal Interface. In: CSREA HCI (2005)
Jurafsky, D., Martin, J.H.: Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition
Hakkani-Tür, D., Tur, G., Riccardi, G., Kim, H.K.: Error Prediction in Spoken Dialog: From Signal-to-Noise Ratio to Semantic Confidence Scores. In: The Proceedings of ICASSP-2005, IEEE International Conference on Acoustics, Speech, and Signal Processing, Philadelphia, USA (March 2005)
Levin, B., Rappaport Hovav, M.: Lexical Semantics and Syntactic Structure. In: Lappin, S. (ed.) The Handbook of Contemporary Semantic Theory, pp. 487–507. Blackwell, Oxford
Foster, M.E.: State of the art review: Multimodal fission (2002), www.hcrc.ed.ac.uk/comic/documents/deliverables/Del6-1.pdfM
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Frattini, G. et al. (2008). Personalized Mobile Multimodal Services: CHAT Project Experiences. In: Song, IY., et al. Advances in Conceptual Modeling – Challenges and Opportunities. ER 2008. Lecture Notes in Computer Science, vol 5232. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87991-6_25
Download citation
DOI: https://doi.org/10.1007/978-3-540-87991-6_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87990-9
Online ISBN: 978-3-540-87991-6
eBook Packages: Computer ScienceComputer Science (R0)