research-article

An object-centric interaction framework for tangible interfaces in virtual environments

Authors:

Johann Habakuk Israel,

Oliver Belaifa,

Adrienne Gispen,

Rainer StarkAuthors Info & Claims

TEI '11: Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction

Pages 325 - 332

https://doi.org/10.1145/1935701.1935777

Published: 22 January 2010 Publication History

Abstract

The spatial and semantic integration of tangible user interfaces (TUI) into virtual environments is a promising approach to enhance mixed reality-applications with dynamic three-dimensional graphics and graspable controls. Although various software frameworks for virtual reality periphery and tangible interaction exist, a novel framework which provides a high-level TUI-object-centric interface (instead of device-centric interface) and duplex access to important physical properties of TUI-objects, e.g. three-dimensional position, sensor states, force actuators could foster the development of such mixed-reality applications.

This paper describes a TUI-VR-framework whose aim is to support the development of physically enriched VR-applications. It focuses on the spatial and manipulative properties of TUI-objects, leaving it to the application to implement interaction techniques, semantics and expressive physical/digital couplings [cf. 10, 27].

On the programming side, the primary goals of the framework are the integration of a device abstraction layer, a lightweight application programming interface and full duplex communication between the TUI-application and interaction devices. The framework allows for a distributed system configuration and is highly customizable. Various virtual reality tracking frameworks and devices (e.g. VRPN, Ascension MotionStar, force-feedback devices) and physical toolkits (e.g. Phidgets) are already integrated. Further adapters can also easily be integrated.

The capabilities and flexibility of the framework are illustrated at the end of the paper by means of two use cases.

References

[1]

Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S. and MacIntyre, B. Recent Advances in Augmented Reality Computer Graphics, 21, 6 (2001), 34--47.

Digital Library

[2]

Billinghurst, M., Grasset, R. and Looser, J. Designing Augmented Reality Interfaces. Copmuter Graphics, 39, 1 (2005), 17--22.

Digital Library

[3]

Billinghurst, M., Kato, H. and Poupyrev, I. MagicBook: Transitioning between Reality and Virtuality. IEEE Computer Grapics and Applications, May/June (2001), 1--4.

Digital Library

[4]

Bowman, D.A., Kruijff, E., LaViola Jr., J.J. and Poupyrev, I. 3D User Interfaces: Theory and Practice. Addison-Wesley, Reading, Massachusetts, 2005.

Digital Library

[5]

Dalheimer, M.K. Programming with QT: Writing portable GUI applications on Unix and Win32. O'Reilly, Sebastopol, 2002.

Digital Library

[6]

Drascic, D. and Milgram, P. Perceptual issues in augmented reality. SPIE Stereoscopic Displays and Applications VII, Virtual Reality Systems III, 2653, (1996), 123--134.

[7]

Dumas, B., Lalanne, D., Guinard, D., Koenig, R. and Ingold, R., Strengths and weaknesses of software architectures for the rapid creation of tangible and multimodal interfaces. In Proc. TEI'08 Second International Conference on Tangible and Embedded Interaction, Bonn (2008), 47--54.

Digital Library

[8]

Greenberg, S. and Fitchett, C., Phidgets: easy development of physical interfaces through physical widgets. In Proc. Symposium on User Interface Software and Technology UIST'01, Orlando, Florida (2001), 209--218.

Digital Library

[9]

Hinckley, K., Pausch, R., Gobble, J. and Kassell, N., Passive real-world interface props for neurosurgical visualization. In Proc. CHI'94, (1994), 452--458.

Digital Library

[10]

Hornecker, E. and Buur, J., Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interaction. In Proc. CHI 2006, (2006), 437--446.

Digital Library

[11]

1Ishii, H., Tangible Bits: Beyond Pixels. In Proc. Second International Conference on Tangible and Embedded Interaction (TEI'08), Bonn, Germany (2008), XV-XXV.

Digital Library

[12]

Ishii, H. and Ullmer, B., Tangible bits: towards seamless interfaces between people, bits, and atoms. In Proc. CHI'97, Atlanta, Georgia (1997), 234--241.

Digital Library

[13]

Israel, J.H., Wiese, E., Mateescu, M. and Stark, R. Investigating three-dimensional sketching for early conceptual design---Results from expert discussions and user studies. Computers & Graphics, 33, 4 (2009), 462--473.

Digital Library

[14]

Jacob, R.J.K., Deligianndis, L. and Morrison, S. A Software Model and Specification Language for Non-WIMP User Interfaces. ACM Transactions on Computer-Human Interaction, 6, 1 (1999), 1--46.

Digital Library

[15]

James, J., Ingalls, T., Qian, G., Olsen, L., Whiteley, D., Wong, S. and Rikakis, T., Movement-based interactive dance performance. In Proc. 14th annual ACM international conference on Multimedia, Santa Barbara, CA, USA (2006), 470--480.

Digital Library

[16]

Kaltenbrunner, M. and Bencina, R., reacTIVision: A Computer-Vision Framework for Table-Based Tangible Interaction. In Proc. First international conference on Tangible and embedded interaction TEI'07, Baton Rouge (2007), 69--74.

Digital Library

[17]

Keefe, D.F., Acevedo Feliz, D., Moscovich, T., Laidlaw, D.H. and LaViola, J., CavePainting: A Fully Immersive 3D Artistic Medium and Interactive Experience. In Proc. ACM Symposium on Interactive 3D Graphics (SI3D'01), (2001), 85--93.

Digital Library

[18]

Klemmer, S.R., Li, J., Lin, J. and Landay, J.A., Papier-Mache: toolkit support for tangible input. In Proc. Conference on Human Factors in Computing Systems, Vienna (2004), 399--406.

Digital Library

[19]

Krause, F.-L., Jansen, H., Kind, C. and Rothenburg, U. Virtual Product Development as an Engine for Innovation. In Krause, F.-L. (ed.) The Future of Product Development. Proceedings of the 17th CIRP Design Conference, Springer-Verlag, Berlin, Heidelberg, New York, 2007, 703--713.

[20]

Lok, B.C. Towards the merging of real and virtual spaces. Communications of the ACM, 47, 8 (2004), 48--53.

Digital Library

[21]

Marquardt, N., Gross, T., Carpendale, S. and Greenberg, S., Revealing the invisible: visualizing the location and event flow of distributed physical devices. In Proc. Fourth international conference on Tangible, embedded, and embodied interaction TEI'10, Cambridge, Massachusetts, USA (2010), 41--48.

Digital Library

[22]

Mazalek, A., Tangible Toolkits: Integrating Application Development across Diverse Multi-User and Tangible Interaction Platforms. In Proc. Let's Get Physical Workshop, DCC'06, Eindhoven (2006).

[23]

Milgram, P. and Drascic, D., Perceptual Effects In Aligning Virtual And Real Objects In Augmented Reality Displays. In Proc. 41st Annual Meeting of Human Factors & Ergonomics Society, Albuquerque, NM, USA (1997), 1239--1243.

[24]

Ohlenburg, J., Broll, W. and Lindt, I. DEVAL -- A Device Abstraction Layer for VR/AR. In Universal Access in Human Computer Interaction. Coping with Diversity, Springer, Berlin, Heidelberg, 2007, 497--506.

Digital Library

[25]

Philip, G.C. Software design guidelines for event-driven programming Journal of Systems and Software, 41, 2 (1998), 79--91.

Digital Library

[26]

Reitmayr, G. and Schmalstieg, D., An open software architecture for virtual reality interaction. In Proc. Symposium on Virtual reality software and technology VRST'01, Baniff, Alberta, Canada (2001), 47--54.

Digital Library

[27]

Shaer, O. and Jacob, R.J.K. A specification paradigm for the design and implementation of tangible user interfaces. ACM Transactions on Computer-Human Interaction (TOCHI), 16, 4 (2009), 1--39.

Digital Library

[28]

Shaer, O., Leland, N., Calvillo-Gamez, E.H. and Jacob, R.J.K. The TAC paradigm: specifying tangible user interfaces. Personal and Ubiquitous Computing, 8, 5 (2004), 359--369.

[29]

Stark, R. and Beckmann-Dobrev, B., Ein interdisziplinärer Ansatz zur multimodalen funktionalen Absicherung mechatronischer Systeme am Beispiel einer PKW-Heckklappe. In Proc. 3. Grazer Symposium Virtuelles Fahrzeug Graz (2010), 18--27.

[30]

Sutherland, I.E., The Ultimate Display. In Proc. IFIP Congress, (1965), 506--508.

[31]

Taylor, R.M., Hudson, T.C., Seeger, A., Weber, H., Juliano, J. and Helser, A.T., VRPN: a device-independent, network-transparent VR peripheral system. In Proc. Symposium on Virtual reality software and technology VRST'01, Baniff, Alberta, Canada (2001), 55--61.

Digital Library

[32]

Ullmer, B. and Ishii, H. Emerging frameworks for tangible user interfaces. In Carroll, J.M. (ed.) Human-computer interaction in the new millennium, Addison-Wesley, Reading, Massachusetts, USA, 2001, 579--601.

[33]

Völlinger, U., Wewetzer, M., Hausser, F. and Ley, A., An Adaptive Multiresolutional Approach for Interactive Simulation of Cloth. In Proc. IASTED Technology Conferences 2010, (2010).

[34]

Wingrave, C.A. and Bowman, D.A., Tiered Developer-Centric Representations for 3D Interfaces: Concept-Oriented Design in Chasm. In Proc. IEEE Virtual Reality, Reno, Nevada, USA (2008), 193--200.

[35]

Woksepp, S. and Olofsson, T. Credibility and applicability of virtual reality models in design and construction. Advanced Engineering Informatics, 22, 4 (2008), 520--528.

Digital Library

Cited By

An DRittenbruch MRazayan LJohal WLemaignan SBrščić DVázquez MCharisi V(2025)The Design of Extended Reality-enabled Tangible Interaction to Enhance the Interaction with Collaborative RobotsProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3721488.3721648(1207-1210)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.5555/3721488.3721648
Ashraf MMarzouk MAmgad NAtia A(2024)Beyond Touch: A Comparative Study of Gesture and Tangible Interactions for Smart Tables Using Multi-Source Data2024 Intelligent Methods, Systems, and Applications (IMSA)10.1109/IMSA61967.2024.10652694(410-415)Online publication date: 13-Jul-2024
https://doi.org/10.1109/IMSA61967.2024.10652694
Neumeyer SExner KKind SHayka HStark R(2017)Virtual Prototyping and Validation of Cpps within a New Software FrameworkComputation10.3390/computation50100105:1(10)Online publication date: 18-Feb-2017
https://doi.org/10.3390/computation5010010
Show More Cited By

Index Terms

An object-centric interaction framework for tangible interfaces in virtual environments

Recommendations

Natural and tangible human-computer interfaces for augmented environments
SIGDOC '08: Proceedings of the 26th annual ACM international conference on Design of communication

In this talk, the author will introduce Augmented Reality as a new media to improve human-computer communication and industrial processes. The author will make a special emphasis in introducing and demonstrating new human-computer interfaces appropriate ...
An approach for enabling the use of immersive virtual reality in desktop hybrid interfaces
IASTED-HCI '07: Proceedings of the Second IASTED International Conference on Human Computer Interaction

Hybrid User Interfaces, which create a heterogeneous environment providing different interaction forms and devices, may be enhanced by exploring more extensively the mixed reality continuum, which ranges from the real world to the complete virtual world,...
One Reality: Augmenting How the Physical World is Experienced by combining Multiple Mixed Reality Modalities
UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology

Most of our daily activities take place in the physical world, which inherently imposes physical constraints. In contrast, the digital world is very flexible, but usually isolated from its physical counterpart. To combine these two realms, many Mixed ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

TEI '11: Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction

January 2011

470 pages

ISBN:9781450304788

DOI:10.1145/1935701

General Chairs:
Mark D. Gross
Carnegie Mellon University / Modular Robotics
,
Nuno Jardim Nunes
University of Madeira
,
Program Chairs:
Ellen Yi-Luen Do
Georgia Tech
,
Stephen Brewster
University of Glasgow
,
Ian Oakley
University of Madeira

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 January 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

TEI'11

Sponsor:

SIGCHI

TEI'11: Fifth International Conference on Tangible, Embedded, and Embodied Interaction

January 22 - 26, 2011

Funchal, Portugal

Acceptance Rates

Overall Acceptance Rate 393 of 1,367 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
876
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)4

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

An DRittenbruch MRazayan LJohal WLemaignan SBrščić DVázquez MCharisi V(2025)The Design of Extended Reality-enabled Tangible Interaction to Enhance the Interaction with Collaborative RobotsProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3721488.3721648(1207-1210)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.5555/3721488.3721648
Ashraf MMarzouk MAmgad NAtia A(2024)Beyond Touch: A Comparative Study of Gesture and Tangible Interactions for Smart Tables Using Multi-Source Data2024 Intelligent Methods, Systems, and Applications (IMSA)10.1109/IMSA61967.2024.10652694(410-415)Online publication date: 13-Jul-2024
https://doi.org/10.1109/IMSA61967.2024.10652694
Neumeyer SExner KKind SHayka HStark R(2017)Virtual Prototyping and Validation of Cpps within a New Software FrameworkComputation10.3390/computation50100105:1(10)Online publication date: 18-Feb-2017
https://doi.org/10.3390/computation5010010
Bellucci AZarraonandia TDíaz PAedo IInakage MIshii HDo ESteimle JShaer OKunze KPeiris R(2017)End-User Prototyping of Cross-Reality EnvironmentsProceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3024969.3024975(173-182)Online publication date: 20-Mar-2017
https://dl.acm.org/doi/10.1145/3024969.3024975
Buchholz CKind SStark R(2017)Design of a Test Environment for Planning and Interaction with Virtual Production ProcessesProcedia CIRP10.1016/j.procir.2016.06.08562(547-552)Online publication date: 2017
https://doi.org/10.1016/j.procir.2016.06.085
Mecke R(2017)Teilprojekt TP 4 – InteraktionAngewandte Virtuelle Techniken im Produktentstehungsprozess10.1007/978-3-662-49317-5_6(223-286)Online publication date: 21-Feb-2017
https://doi.org/10.1007/978-3-662-49317-5_6
Hoffmann H(2017)Teilprojekt TP 2 – Simulation und RenderingAngewandte Virtuelle Techniken im Produktentstehungsprozess10.1007/978-3-662-49317-5_4(57-140)Online publication date: 21-Feb-2017
https://doi.org/10.1007/978-3-662-49317-5_4
Bao RCai H(2014)Management of complex data objects in ship designing process2014 International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA.2014.7058123(534-540)Online publication date: Oct-2014
https://doi.org/10.1109/DSAA.2014.7058123
Mott MDonahue TPoor GLeventhal LKonstan JChi EHöök K(2012)Leveraging motor learning for a tangible password systemCHI '12 Extended Abstracts on Human Factors in Computing Systems10.1145/2212776.2223842(2597-2602)Online publication date: 5-May-2012
https://dl.acm.org/doi/10.1145/2212776.2223842
Adenauer JIsrael JStark R(2012)Virtual Reality Technologies for Creative DesignCIRP Design 201210.1007/978-1-4471-4507-3_13(125-135)Online publication date: 20-Nov-2012
https://doi.org/10.1007/978-1-4471-4507-3_13
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten