Abstract
This chapter describes a framework to integrate voice interaction in 3D worlds allowing users to manage VRML objects by using speech dialogs. We have defined a language named XMMVR to specify in a common program the 3D scenes and the multimodal interaction. XMMVR is based on the theater metaphor adding the possibility to include speech dialogs for the user to control the 3D action. This language is based on the XML standard reusing other standard languages such as VRML for graphics and VoiceXML for speech dialogs. We also describe a platform to support XMMVR that integrates the speech dialog manager, GUI interaction (graphical output and mouse input), task division, and event management.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sherman, W.R., Craig, A., Understanding Virtual Reality: Interface, Application, and Design, The Morgan Kaufmann Series in Computer Graphics, 2002
Dahl, D. Practical Spoken Dialog Systems (Text, Speech and Language Technology), Springer, 2004
Bolt, R.A., “Put-That-There”: Voice and Gesture at the Graphics Interface ACM Siggraph Computer Graphics, 1980
Cohen, P., Oviatt, S., The Role of Voice Input for Human-Machine Communication Proceedings of the National Academy of Sciences, 1994
González-Ferreras, C., González Escribano, A., Escudero Mancebo, D., V. Cardeñoso Payo. Incorporación de interacción vocal en mundos virtuales usando VoiceXML, CEIG, 2004
Bowman, D. A., Kluijff, E., Laviola, J., Poupyrev I., 3d User Interfaces. Theory and Practice. Addison Wesley 2005
McGlashan, S., Axling, T., Talking to Agents in Virtual Worlds, UK VR-SIG Conf., 1996
VoiceXML Forum. “Voice eXtensible Markup Language”: http://www.voicexml.org (Last access: June 2008)
Extensible 3D (X3D): http://www.web3d.org (Last access: June 2008)
Hartman, J., Wernecke, J., The VRML 2.0 Handbook, Silicon Graphics, 1994
SALT Technical White Paper: http://www.saltforum.org/whitepapers/whitepapers.asp (Last access: June 2008)
XHTML+Voice Profile 1.2: http://www.voicexml.org/specs/multimodal/x+v/12/spec.html (Revised at December 2007)
R. Dachselt. BEHAVIOR3D: An XML-Based Framework for 3D Graphics Behavior; ACM Web3D, 2003
VHML Standard: http://www.vhml.org (Last access: June 2008)
Latoschik, M.E. Designing transition networks for multimodal VR-interactions using a markup language, ICMI, 2002
Okazaki, N. et al. An Extension of the Multimodal Presentation Markup Language (MPML) to a Three-Dimensional VRML Space, Wiley-Interscience 2005
Carretero, M.P. et al. Animación Facial y Corporal de Avatares 3D a partir de la edición e interpretación de lenguajes de marcas, CEIG, 2004
XMMVR DTD: http://verbo.dcs.fi.uva.es/holmedo/xmmvr/xmmvr.dtd
CORTONA: http://www.parallelgraphics.com/products/cortona/ (Last access: June 2008)
Phelps, A.M. Introduction to the External Authoring Interface, EAI. Rochester Institute of Technology, Department of Information Technology, http://andysgi.rit.edu/andyworld10/gallery/archives/vrml/media/eaiclass.doc (Revised at December 2006)
ATLAS IBERVOX: http://www.verbio.com (Last access: June 2008)
Multimodal Architecture and Interfaces: http://www.w3.org/TR/mmi-arch (Last access: June 2008)
COLLADA: http://www.collada.org (Last access: June 2008)
NICE: http://www.niceproject.com (Last access: June 2008)
CONTIGRA: http://www-mmt.inf.tu-dresden.de/Forschung/Projekte/CONTIGRA/index_en. xhtml (Revised at December 2007)
SAI, Scene Access Interface: http://www.xj3d.org/tutorials/general_sai.html (Last access: June 2008)
FreeWRL: http://freewrl.sourceforge.net (Last access: June 2008)
XJ3D: http://www.xj3d.org (Last access: June 2008)
Acknowledgment
This work has been partially financed by the research project of the Junta de Castilla y León VA077A08.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag London Limited
About this chapter
Cite this chapter
Olmedo-Rodríguez, H., Escudero-Mancebo, D., Cardeñoso-Payo, V., González-Ferreras, C., González-Escribano, A. (2009). Conceptual and Practical Framework for the Integration of Multimodal Interaction in 3D Worlds. In: Macías, J., Granollers Saltiveri, A., Latorre, P. (eds) New Trends on Human–Computer Interaction. Springer, London. https://doi.org/10.1007/978-1-84882-352-5_9
Download citation
DOI: https://doi.org/10.1007/978-1-84882-352-5_9
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-84882-351-8
Online ISBN: 978-1-84882-352-5
eBook Packages: Computer ScienceComputer Science (R0)