Skip to main content
SpringerLink
Log in

IMMIView: a multi-user solution for design review in real-time

  • Special Issue
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

IMMIView is an interactive system that relies on multiple modalities and multi-user interaction to support collaborative design review. It was designed to offer natural interaction in visualization setups such as large-scale displays, head mounted displays or TabletPC computers. To support architectural design, our system provides content creation and manipulation, 3D scene navigation and annotations. Users can interact with the system using laser pointers, speech commands, body gestures and mobile devices. In this paper, we describe how we design a system to answer architectural user requirements. In particular, our system takes advantage of multiple modalities to provide a natural interaction for design review. We also propose a new graphical user interface adapted to architectural user tasks, such as navigation or annotations. The interface relies on a novel stroke-based interaction supported by simple laser pointers as input devices for large-scale displays. Furthermore, input devices such as speech and body tracking allow IMMIView to support multiple users. Moreover, they allow each user to select different modalities according to their preference and modality adequacy for the user task. We present a multi-modal fusion system developed to support multi-modal commands on a collaborative, co-located, environment, i.e. with two or more users interacting at the same time, on the same system. The multi-modal fusion system listens to inputs from all the IMMIView modules in order to model user actions and issue commands. The multiple modalities are fused based on a simple rule-based sub-module developed in IMMIView and presented in this paper. User evaluation performed over IMMIView is presented. The results show that users feel comfortable with the system and suggest that users prefer the multi-modal approach to more conventional interactions, such as mouse and menus, for the architectural tasks presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. de Araùjo, B., Jorge, J.: Blobmaker: Free-form modelling with variational implicit surfaces. In: 12 Encontro Português de Computação Gráfica, pp. 335–342 (2003)

  2. Bae, S.H., Kobayash, T., Kijima, R., Kim, W.S.: Tangible nurbs-curve manipulation techniques using graspable handles on a large display. In: UIST ’04. Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology, pp. 81–90. ACM Press, New York (2004)

  3. Broll, W.: The augmented round table—a new interface to urban planning and architectural design. In: Rauterberg M., Menozzi M., Wesson J. (eds.) INTERACT. IOS Press, Amsterdam (2003)

  4. Buxton, W., Fitzmaurice, G., Balakrishnan, R., Kurtenbach, G.: Large displays in automotive design. IEEE Comput Graph Appl 20(4), 68–75 (2000)

    Article  Google Scholar 

  5. Callahan, J., Hopkins, D., Weiser, M., Shneiderman, B.: An empirical comparison of pie vs. linear menus. In: CHI ’88. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 95–100. ACM Press, New York (1988)

  6. Cao, X., Balakrishnan, R.: Visionwand: interaction techniques for large displays using a passive wand tracked in 3d. ACM Trans Graph 23(3), 729–729 (2004)

    Article  Google Scholar 

  7. Davis, J., Chen, X.: Lumipoint: multi-user laser-based interaction on large tiled displays. Displays 23(5) (2002)

  8. Deering, M.F.: Holosketch: a virtual reality sketching/animation tool. ACM Trans Comput Hum Interact 2(3), 220–238 (1995)

    Article  Google Scholar 

  9. Drettakis, G., Roussou, M., Reche, A., Tsingos, N.: Design and evaluation of a real-world virtual environment for architecture and urban planning. Presence: Teleoper Virtual Environ 16(3), 318–332 (2007)

    Article  Google Scholar 

  10. Dvorak, J., Hamata, V., Skacilik, J., Benes, B.: Boosting up architectural design education with virtual reality. In: CEMVR’05. Central European Multimedia and Virtual Reality Conference, Eurographics, pp. 95–200 (2005)

  11. Fonseca, M.J., Pimentel, C., Jorge, J.A.: CALI: an online scribble recognizer for calligraphic interfaces. In: Proceedings of the 2002 AAAI Spring Symposium—Sketch Understanding, pp. 51–58. Palo Alto (2002)

  12. Grossman, T., Balakrishnan, R., Kurtenbach, G., Fitzmaurice, G., Khan, A., Buxton, B.: Creating principal 3d curves with digital tape drawing. In: CHI ’02. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 121–128. ACM Press, New York (2002)

  13. Hopkins, D.: The design and implementation of pie menus. Dr Dobb’s J 16(12), 16–26 (1991)

    Google Scholar 

  14. Hur, H., Fleisch, T., Kim, T.B., On, G.: Aici-advanced immersive collaborative interaction framework. In: Proceedings of the SCCE CAD/CAM Conference: Book of Abstracts & Proceedings CD-ROM., Society of CAD/CAM Engineers (SCCE), pp. 20–25. Pyungchang, Korea (2006)

  15. Iacucci, G., Wagner, I.: Supporting collaboration ubiquitously: an augmented learning environment for architecture students. In: ECSCW’03. Proceedings of the Eighth Conference on European Conference on Computer Supported Cooperative Work, pp. 139–158. Kluwer Academic Publishers, Norwell (2003)

  16. Igarashi, T., Matsuoka, S., Tanaka, H.: Teddy: A sketching interface for 3d freeform design. Proceedings of SIGGRAPH 99, pp. 409–416, ISBN 0-20148-560-5. Los Angeles (1999)

  17. IGD, F., ZGDV.: Instantreality: advanced mixed reality technology. http://www.instantreality.org/ (2009). Retrieved Oct 2009

  18. Isenberg, T., Neumann, P., Carpendale, S., Nix, S., Greenberg, S.: Interactive annotations on large, high-resolution information displays. In: 2006 Conference Compendium of IEEE VIS, IEEE InfoVis, and IEEE VAST (Baltimore, MA, USA), pp. 124–125. IEEE Computer Society, Los Alamitos (2006) (extended abstract and poster)

  19. Ishii, H., Ben-Joseph, E., Underkoffler, J., Yeung, L., Chak, D., Kanji, Z., Piper, B.: Augmented urban planning workbench: overlaying drawings, physical models and digital simulation. In: ISMAR ’02. Proceedings of the 1st International Symposium on Mixed and Augmented Reality, p. 203. IEEE Computer Society, Washington, DC (2002)

  20. Jacoby, R.H., Ellis, S.R.: Using virtual menus in a virtual environment. In: Proceedings of SPIE, Visual Data Interpretation (1992)

  21. Jung, T., Gross, M.D., Do, E.Y.L.: Sketching annotations in a 3d web environment. In: CHI ’02. CHI ’02 Extended Abstracts on Human Factors in Computing Systems, pp. 618–619. ACM, New York (2002)

  22. Kadobayashi, R., Lombardi, J., McCahill, M.P., Stearns, H., Tanaka, K., Kay, A.: 3d model annotation from multiple viewpoints for croquet. In: C5 ’06. Proceedings of the Fourth International Conference on Creating, Connecting and Collaborating through Computing, pp. 10–15. IEEE Computer Society, Washington, DC (2006)

  23. Kim, M., Maher, M.L.: Comparison of designers using a tangible user interface and a graphical user interface and the impact on spatial cognition. In: Proceedings of International Workshop on Human Behaviour in Designing, Key Centre of Design Computing and Cognition, pp. 81–94. University of Sydney, Australia (2005)

  24. Kleinermann, F., Troyer, O.D., Creelle, C., Pellens, B.: Adding semantic annotations, navigation paths and tour guides for existing virtual environments. In: Proceedings of the 13th International Conference on Virtual Systems and Multimedia, pp. 50–62. Springer, Brisbane (2007)

  25. Lapointe, J.F., Godin, G.: On-screen laser spot detection for large display interaction. In: Haptic Audio Visual Environments and their Applications, p. 5. IEEE International Workshop (2005)

  26. Nealen, A., Sorkine, O., Alexa, M., Cohen-Or, D.: A sketch-based interface for detail-preserving mesh editing. In: SIGGRAPH ’05: ACM SIGGRAPH 2005 Papers, pp. 1142–1147. ACM Press, New York (2005)

  27. Oh, J.Y., Stuerzlinger, W.: Laser pointers as collaborative pointing devices. In: Proceedings of Graphics Interface, pp. 141–150 (2002)

  28. OpenSG.: Opensg. Opensource scenegraph. http://www.opensg.org (2009)

  29. Pereira, J.P., Jorge, J.A., Branco, V.A., Ferreira, F.N.: Calligraphic interfaces: mixed metaphors for design. In: Jorge J.A., Nunes N.J., Cunha, J.F. (eds.) DSV-IS, Lecture Notes in Computer Science, vol 2844, pp. 154–170. Springer, Berlin (2003)

  30. Reitmayr, G., Schmalstieg, D.: Opentracker: a flexible software design for three-dimensional interaction. Virtual Real 9(1),79–92 (2005)

    Article  Google Scholar 

  31. Schmalstieg, D., Fuhrmann, A., Hesina, G., Szalavári, Z., Encarnaçäo, L.M., Gervautz, M., Purgathofer, W.: The studierstube augmented reality project. Presence Teleoper Virtual Environ 11(1), 33–54 (2002)

    Article  Google Scholar 

  32. Shesh, A., Chen, B.: Smartpaper: an interactive and user friendly sketching system. Comput Graph Forum 23(3), 301–310 (2004)

    Article  Google Scholar 

  33. Wang, X.: Specifying augmented virtuality systems for creative architectural design. In: IV ’07: Proceedings of the 11th International Conference Information Visualization, pp. 584–589. IEEE Computer Society, Washington, DC (2007)

  34. Welch, G., Bishop, G.: An introduction to the kalman filter. Technical report. University of North Carolina, USA (2006)

  35. XMLBlaster.: Xmlblaster. Open source for message oriented middleware. http://www.xmlblaster.org (2007)

  36. Zeleznik, R.C., Herndon, K.P., Hughes, J.F.: SKETCH: an Interface for sketching 3D scenes. In: SIGGRAPH 96 Conference Proceedings, pp. 163–170 (1996)

Download references

Acknowledgments

Ricardo Jota was supported by the Portuguese Foundation for Science and Technology, Grant reference SFRH/BD/17574/2004. Bruno Araújo was supported by the Portuguese Foundation for Science and Technology, Grant reference SFRH/BD/31020/2006. We would like to thank José Pedro Dias for his work on IMMIView.

Author information

Authors and Affiliations

  1. VIMMI Group, Department of Computer Science and Engineering, INESC-ID, IST/Technical University of Lisbon, Rua Alves Redol, 9, 1000-029, Lisboa, Portugal

    Ricardo Jota, Bruno R. de Araújo, Luís C. Bruno, João M. Pereira & Joaquim A. Jorge

Authors
  1. Ricardo Jota
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruno R. de Araújo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luís C. Bruno
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. João M. Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Joaquim A. Jorge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricardo Jota.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jota, R., de Araújo, B.R., Bruno, L.C. et al. IMMIView: a multi-user solution for design review in real-time. J Real-Time Image Proc 5, 91–107 (2010). https://doi.org/10.1007/s11554-009-0141-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-009-0141-1

Keywords

Search

Navigation