skip to main content
10.1145/1452392.1452418acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
research-article

Designing context-aware multimodal virtual environments

Published: 20 October 2008 Publication History

Abstract

Despite of decades of research, creating intuitive and easy to learn interfaces for 3D virtual environments (VE) is still not obvious, requiring VE specialists to define, implement and evaluate solutions in an iterative way, often using low-level programming code. Moreover, quite frequently the interaction with the virtual environment may also vary dependent on the context in which it is applied, such as the available hardware setup, user experience, or the pose of the user (e.g. sitting or standing). Lacking other tools, the context-awareness of an application is usually implemented in an ad-hoc manner, using low-level programming, as well. This may result in code that is difficult and expensive to maintain. One possible approach to facilitate the process of creating these highly interactive user interfaces is by adopting a model-based user interface design. This lifts the creation of a user interface to a higher level allowing the designer to reason more in terms of high-level concepts, rather than writing programming code. In this paper, we adopt a model-based user interface design (MBUID) process for the creation of VEs, and explain how a context system using an Event-Condition-Action paradigm is added. We illustrate our approach by means of a case study.

References

[1]
M. Baldauf, S. Dustdar, and F. Rosenberg. A survey on context-aware systems. International Journal of Ad Hoc and Ubiquitous Computing, 2(4):263--277, 2007.
[2]
W. Beer, V. Christian, A. Ferscha, and L. Mehrmann. Modeling Context-aware Behavior by Interpreted ECA Rules. volume 2790, pages 1064--1073. Springer.
[3]
D. A. Bowman, E. Kruijiff, J. J. LaViola, and I. Poupyrev. 3D User Interfaces, Theory and Practice. Addison-Wesley, 2005.
[4]
D. Carr. Interaction object graphs: An executable graphical notation for specifying user interfaces. In Formal Methods for Computer-Human Interaction, pages 141--156. Springer-Verlag, 1997.
[5]
T. Clerckx. Model-Based Development of Context-Aware Interactive Applications in Ambient Intelligence Environments. PhD thesis, transnationale Universiteit Limburg, June 2007.
[6]
K. Coninx, E. Cuppens, J. De Boeck, and C. Raymaekers. Integrating support for usability evaluation into high level interaction descriptions with NiMMiT. In Proceedings of 13th International Workshop on Design, Specification and Verification of Interactive Systems (DSVIS'06), volume 4385, Dublin, Ireland, July 26-28 2006.
[7]
J. Coutaz and G. Rey. Foundations for a Theory of Contextors. In C. Kolski and J. Vanderdonckt, editors, Computer-Aided Design of User Interfaces III, volume 3, pages 13--33. Kluwer Academic, 2002. Invited talk.
[8]
J. De Boeck, C. Raymaekers, and K. Coninx. A tool supporting model based user interface design in 3d virtual environments. In Proceedings of the International Conference on Computer Graphics Theory and Applications (GRAPP08), Funchal, Portugal, January 22-25 2008.
[9]
J. De Boeck, D. Vanacken, C. Raymaekers, and K. Coninx. High-level modeling of multimodal interaction techniques using nimmit. Journal of Virtual Reality and Broadcasting, 4(2), September 2007. urn:nbn:de:0009-6-11615.
[10]
A. K. Dey. Providing Architectural Support for Building Context-Aware Applications. PhD thesis, College of Computing, Georgia Institute of Technology, Dec. 2000.
[11]
P. Dragicevic and J.-D. Fekete. Support for input adaptability in the ICON toolkit. In Proceedings of the 6th international conference on multimodal interfaces (ICMI04), pages 212--219, State College, PA, USA, 2004.
[12]
B. Dumas, D. Lalanne, D. Guinard, R. Koenig, and R. Ingold. Strengths and weaknesses of software architectures for the rapid creation of tangible and multimodal interfaces. In TEI '08: Proceedings of the 2nd international conference on Tangible and embedded interaction, pages 47--54, 2008.
[13]
R. Etter, P. Costa, and T. Broens. A Rule-Based Approach Towards Context-Aware User Notification Services. pages 281--284, 2006.
[14]
V. Fernandes, T. Guerreiro, B. Araújo, J. Jorge, and a. P. Jo. Extensible middleware framework for multimodal interfaces in distributed environments. In ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces, pages 216--219, 2007.
[15]
P. Figueroa, M. Green, and H. J. Hoover. InTml: A description language for VR applications. In Proceedings of Web3D'02, pages 53--58, Arizona, USA, Februari 2002.
[16]
F. Flippo, A. Krebs, and I. Marsic. A framework for rapid development of multimodal interfaces. In ICMI '03: Proceedings of the 5th international conference on Multimodal interfaces, pages 109--116, 2003.
[17]
S. Irawati, S. Ahn, J. Kim, and H. Ko. VARU Framework: Enabling Rapid Prototyping of VR, AR and Ubiquitous Applications. In Virtual Reality Conference, 2008. VR'08. IEEE, pages 201--208, 2008.
[18]
R. Kernchen, P. Boda, K. Moessner, B. Mrohs, M. Boussard, and G. Giuliani. Multimodal user interfaces for context-aware mobile applications. In 16th Annual IEEE International Symposium on Personal Indoor and Mobile Radio Communications (PIMRC), pages 2268--2273, 2005.
[19]
J. Lee, G. Rhee, H. Kim, K. Lee, Y. Suh, and K. Kim. Convergence of Context-Awareness and Augmented Reality for Ubiquitous Services and Immersive Interactions. In Computational Science and Its Applications - ICCSA 2006, pages 466--474. Springer, 2006.
[20]
C. Li and K. Willis. Modeling context aware interaction for wayfinding using mobile devices. In MobileHCI '06: Proceedings of the 8th conference on Human-computer interaction with mobile devices and services, pages 97--100, 2006.
[21]
B. MacIntyre and S. Feiner. Language-level support for exploratory programming of distributed virtual environments. In UIST '96: Proceedings of the 9th annual ACM symposium on User interface software and technology, pages 83--94, 1996.
[22]
F. V. Mario Gutiérrez, Daniel Thalmann. Semantic virtual environments with adaptive multimodal interfaces. In 11th International Conference on Multimedia Modelling, MMM2005, pages 277--283, 2005.
[23]
D. Navarre, P. Palanque, R. Bastide, A. Schyn, M. Winckler, L. Nedel, and C. Freitas. A formal description of multimodal interaction techniques for immersive virtual reality applications. In Proceedings of Tenth IFIP TC13 International Conference on Human-Computer Interaction, Rome, IT, September 12-16 2005.
[24]
J. M. S. O., J. Serrat, K. Yang, and E. S. C. Modelling context information for managing pervasive network services. In Proc. of the International Conference on Modelling and Simulation (ICMS '05), pages 35--39, 2005.
[25]
D. Preuveneers, J. Van den Bergh, D. Wagelaar, A. Georges, P. Rigole, T. Clerckx, Y. Berbers, K. Coninx, V. Jonckers, and K. D. Bosschere. Towards an Extensible Context Ontology for Ambient Intelligence. In P. Markopoulos, B. Eggen, E. Aarts, and J. L. Crowley, editors, Second European Symposium on Ambient Intelligence, volume 3295 of LNCS, pages 148--59, Eindhoven, The Netherlands, Nov 8 - 11 2004. Springer.
[26]
C. Raymaekers, K. Coninx, J. D. Boeck, E. Cuppens, and E. Flerackers. High-vel interaction modelling to facilitate the development of virtual environments. 2004 May 12-14. Proceedings of Virtual Reality International Conference, Laval, FR.
[27]
C. Rousseau, Y. Bellik, and F. Vernier. Multimodal output specification / simulation platform. In ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces, pages 84--91, 2005.
[28]
M. Serrano, L. Nigay, J.-Y. L. Lawson, A. Ramsay, R. Murray-Smith, and S. Denef. The openinterface framework: a tool for multimodal interaction. In CHI '08: CHI '08 extended abstracts on Human factors in computing systems, pages 3501--3506, 2008.
[29]
T. Sohn and A. K. Dey. icap: an informal tool for interactive prototyping of context-aware applications. In CHI Extended Abstracts, pages 974--975, 2003.
[30]
L. Vanacken, E. Cuppens, T. Clerckx, and K. Coninx. Extending a dialog model with contextual knowledge. In M. Winckler, H. Johnson, and P. A. Palanque, editors, TAMODIA, volume 4849 of Lecture Notes in Computer Science, pages 28--41. Springer, 2007.
[31]
J. Willans and M. Harrison. A toolset supported approach for designing and testing virtual environment interaction techniques. International Journal of Human-Computer Studies, 55(2):145--165, August 2001.

Cited By

View all
  • (2015)Immersiveness of Ubiquitous Computing Environments PrototypesProceedings of the Third International Conference on Distributed, Ambient, and Pervasive Interactions - Volume 918910.1007/978-3-319-20804-6_22(237-248)Online publication date: 2-Aug-2015
  • (2014)Prototyping and analysing ubiquitous computing environments using multiple layersInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2014.02.00172:5(488-506)Online publication date: May-2014
  • (2009)Facilitating adaptation in virtual environments using a context-aware model-based design processProceedings of the 8th international conference on Task Models and Diagrams for User Interface Design10.1007/978-3-642-11797-8_5(58-71)Online publication date: 23-Sep-2009
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI '08: Proceedings of the 10th international conference on Multimodal interfaces
October 2008
322 pages
ISBN:9781605581989
DOI:10.1145/1452392
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2008

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. context-awareness
  2. model-based user interface design
  3. multimodal interaction techniques

Qualifiers

  • Research-article

Conference

ICMI '08
Sponsor:
ICMI '08: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERFACES
October 20 - 22, 2008
Crete, Chania, Greece

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2015)Immersiveness of Ubiquitous Computing Environments PrototypesProceedings of the Third International Conference on Distributed, Ambient, and Pervasive Interactions - Volume 918910.1007/978-3-319-20804-6_22(237-248)Online publication date: 2-Aug-2015
  • (2014)Prototyping and analysing ubiquitous computing environments using multiple layersInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2014.02.00172:5(488-506)Online publication date: May-2014
  • (2009)Facilitating adaptation in virtual environments using a context-aware model-based design processProceedings of the 8th international conference on Task Models and Diagrams for User Interface Design10.1007/978-3-642-11797-8_5(58-71)Online publication date: 23-Sep-2009
  • (2009)CoGenIVE: Building 3D Virtual Environments Using a Model Based User Interface Design ApproachComputer Vision and Computer Graphics. Theory and Applications10.1007/978-3-642-10226-4_7(83-96)Online publication date: 2009

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media