Abstract
This article describes the integration of knowledge based techniques into simulative Virtual Reality (VR) applications. The approach is motivated using multimodal Virtual Construction as an example domain. An abstract Knowledge Representation Layer (KRL) is proposed which is expressive enough to define all necessary data for diverse simulation tasks and which additionally provides a base formalism for the integration of Artificial Intelligence (AI) representations. The KRL supports two different implementation methods. The first method uses XSLT processing to transform the external KRL format into the representation formats of the diverse target systems. The second method implements the KRL using a functionally extendable semantic network. The semantic net library is tailored for real time simulation systems where it interconnects the required simulation modules and establishes access to the knowledge representations inside the simulation loop. The KRL promotes a novel object model for simulated objects called Semantic Entities which provides a uniform access to the KRL and which allows extensive system modularization. The KRL approach is demonstrated in two simulation areas. First, a generalized scene graph representation is presented which introduces an abstract definition and implementation of geometric node interrelations. It supports scene and application structures which can not be expressed using common scene hierarchies or field route concepts. Second, the KRL’s expressiveness is demonstrated in the design of multimodal interactions. Here, the KRL defines the knowledge particularly required during the semantic analysis of multimodal user utterances.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Biermann, P., Jung, B.: Variant design in immersive virtual reality: A markup language for scalable CSG parts. In: Perales, F.J., Draper, B.A. (eds.) AMDO 2004. LNCS, vol. 3179, pp. 123–133. Springer, Heidelberg (2004)
Biermann, P., Wachsmuth, I.: Non-physical simulation of gears and modifiable connections in virtual reality. In: Proceedings Fifth Virtual Reality International Conference (VRIC 2003), Laval, France, pp. 159–164 (2003)
Carey, R., Bell, G., Marrin, C.: Iso/iec 14772-1:1997 virtual reality modeling language (vrml). The VRML Consortium Incorporated (1997)
Cavazza, M., Palmer, I.: High-level interpretation in dynamic virtual environments. Applied Artificial Intelligence 14(1), 125–144 (2000)
Diehl, S., Keller, J.: Constraints for 3D graphics on the internet. In: Proceedings of 5th International Conference on Computer Graphics and Artificial Intelligence 3IA’2002, Limoges, France (2002)
Fernando, T., Marcelino, L., Wimalaratne, P.: Constraint-based immersive virtual environment for supporting assembly and maintenance task. In: Human Computer Interaction International 2001, New Orleans, USA (2001)
Kopp, S., Jung, B., Lessmann, N., Wachsmuth, I.: Max—a multimodal assistant in virtual reality construction. KI-Künstliche Intelligenz 03(4), 11–17 (2003)
Latoschik, M.E.: A gesture processing framework for multimodal interaction in virtual reality. In: Chalmers, A., Lalioti, V. (eds.) AFRIGRAPH 2001, 1st International Conference on Computer Graphics, Virtual Reality and Visualisation in Africa, pp. 95–100. ACM Press, New York (2001)
Latoschik, M.E.: Designing transition networks for multimodal VR-interactions using a markup language. In: Fourth IEEE International Conference on Multimodal Interfaces ICMI 2002, Pittsburgh, Pennsylvania, pp. 411–416. IEEE Press, Los Alamitos (2002)
Latoschik, M.E., Schilling, M.: Incorporating VR databases into AI knowledge representations: A framework for intelligent graphics applications. In: Sixth IASTED International Conference on Computer Graphics and Imaging, pp. 79–84. ACTA Press (2003)
Luck, M., Aylett, R.: Applying artificial intelligence to virtual reality: Intelligent virtual environments. Applied Artificial Intelligence 14(1), 3–32 (2000)
Peters, S., Shrobe, H.: Using semantic networks for knowledge representation in an intelligent environment. In: PerCom 2003: 1st Annual IEEE International Conference on Pervasive Computing and Communications, Ft. Worth, TX, USA. IEEE, Los Alamitos (2003)
Pfeiffer, T., Latoschik, M.E.: Resolving object references in multimodal dialogues for immersive virtual environments. In: IEEE VR2004, Chicago, pp. 35–42. IEEE, Los Alamitos (2004)
Soto, M., Allongue, S.: Modeling methods for reusable and interoperable virtual entities in multimedia virtual worlds. Multimedia Tools Appl. 16(1-2), 161–177 (2002)
Tramberend, H.: Avocado: A distributed virtual reality framework. In: 1999 IEEE Virtual Reality Conference (VR 1999), Houston, Texas, pp. 14–21. IEEE, Los Alamitos (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Latoschik, M.E., Biermann, P., Wachsmuth, I. (2005). Knowledge in the Loop: Semantics Representation for Multimodal Simulative Environments. In: Butz, A., Fisher, B., Krüger, A., Olivier, P. (eds) Smart Graphics. SG 2005. Lecture Notes in Computer Science, vol 3638. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11536482_3
Download citation
DOI: https://doi.org/10.1007/11536482_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28179-5
Online ISBN: 978-3-540-31905-4
eBook Packages: Computer ScienceComputer Science (R0)