research-article

Immersive mixed reality object interaction for collaborative context-aware mobile training and exploration

Authors:

Steffen RothkugelAuthors Info & Claims

MMVE '19: Proceedings of the 11th ACM Workshop on Immersive Mixed and Virtual Environment Systems

Pages 4 - 9

https://doi.org/10.1145/3304113.3326117

Published: 18 June 2019 Publication History

Abstract

The recent generation of mobile devices brings the necessary processing power, sensorics and other hardware to enable highly interactive and immersive mixed reality experiences. However, existing applications mainly focus on single users or the individual experience, and are often limited to basic user-device interaction.

This paper discusses an immersive object interaction approach based on gestural input generated solely from the integrated camera of a mobile device. Developed within the context of the CollaTrEx framework for collaborative context-aware mobile training and exploration, it particularly allows for in-situ multiuser interaction with virtual objects. The presented approach is demonstrated in a set of prototypical game implementations for smartphones.

References

[1]

Benjamin Bach, Ronell Sicat, Johanna Beyer, Maxime Cordeil, and Hanspeter Pfister. 2018. The Hologram in My Hand: How Effective is Interactive Exploration of 3D Visualizations in Immersive Tangible Augmented Reality? IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 457--456.

[2]

Herbert Bay, Tinne Tuytelaars, and Luc" Van Gool. 2006. SURF: Speeded Up Robust Features. In Proceedings of the 9th European Conference on Computer Vision (ECCV'06). 404--417.

Digital Library

[3]

Mark Billinghurst and Hirokazu Kato. 2002. Collaborative Augmented Reality. Commun. ACM 45, 7 (2002), 64--70.

Digital Library

[4]

Jean Botev. 2016. Demonstrating the CollaTrEx Framework for Collaborative Context-Aware Mobile Training and Exploration. In Proceedings of the 4th International Conference on Educational Technologies (ICEduTech'16). 355--356.

[5]

Jean Botev, Ralph Marschall, and Steffen Rothkugel. 2016. CollaTrEx - Collaborative Context-Aware Mobile Training and Exploration. In Proceedings of the 1st International Conference on Design, Learning & Innovation (DLI'16). 113--120.

[6]

Jean Botev, Steffen Rothkugel, and Joe Mayer. 2017. Contour Drawing and Detection for Collaborative Context-Aware Mobile Training and Exploration. In Proceedings of the 16th World Conference on Mobile and Contextual Learning (mLearn'17). 1--4.

Digital Library

[7]

Gary Bradski. 2000. The OpenCV Library. Dr Dobb's Journal of Software Tools 25 (2000), 120--125.

[8]

Wendy H. Chun and Tobias Höllerer. 2013. Real-time Hand Interaction for Augmented Reality on Mobile Phones. In Proceedings of the International Conference on Intelligent User Interfaces (IUI'13). 307--314.

Digital Library

[9]

Anders Henrysson and Mark Ollila. 2004. UMAR: Ubiquitous Mobile Augmented Reality. In Proceedings of the 3rd International Conference on Mobile and Ubiquitous Multimedia (MUM'04). 41--45.

Digital Library

[10]

Wolfgang Hürst and Casper van Wezel. 2013. Gesture-Based Interaction via Finger Tracking for Mobile Augmented Reality. Multimedia Tools and Applications 62, 1 (2013), 233--258.

[11]

Taehee Lee and Tobias Höllerer. 2007. Handy AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking. In Proceedings of the 11th IEEE International Symposium on Wearable Computers (ISWC'07). 83--90.

Digital Library

[12]

David G. Lowe. 2004. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60, 2 (2004), 91--110.

Digital Library

[13]

Paul Milgram and Fumio Kishino. 1994. A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information Systems E77-D, 12 (1994), 1321--1329.

[14]

Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2011. ORB: an Efficient Alternative to SIFT or SURF. In Proceedings of the 13th International Conference on Computer Vision (ICCV'11). 2564--2571.

Digital Library

[15]

Dieter Schmalstieg and Tobias Höllerer. 2016. Augmented Reality: Principles and Practice. Addison-Wesley.

Digital Library

[16]

Jack Sklansky. 1982. Finding the Convex Hull of a Simple Polygon. Pattern Recognition Letters 1, 2 (1982), 79--83.

Digital Library

[17]

Satoshi Suzuki and Keiichi Abe. 1985. Topological Structural Analysis of Digitized Binary Images by Border Following. Computer Vision, Graphics, and Image Processing 30, 1 (1985), 32--46.

[18]

Alexandros Vasilou and Anastasios A. Economides. 2007. Mobile Collaborative Learning Using Multicast MANETs. International Journal of Mobile Communications 5, 4 (2007), 423--444.

Digital Library

[19]

Daniel Wagner, Thomas Pintaric, Florian Ledermann, and Dieter Schmalstieg. 2005. Towards Massively Multi-user Augmented Reality on Handheld Devices. In Proceedings of the 3rd International Conference on Pervasive Computing (PERVASIVE'05). 208--219.

Digital Library

[20]

Shahrouz Yousefi, Mhretab Kidane, Yeray Delgado, Julio Chana, and Nico Reski. 2016. 3D Gesture-Based Interaction for Immersive Experience in Mobile VR. In Proceedings of the 23rd International Conference on Pattern Recognition (ICPR'16). 2121--2126.

Cited By

Damaševičius RSidekerskienė T(2025)Technology in Digital Escape RoomsDigital Escape Room Designs in Education10.4018/979-8-3693-4219-0.ch003(107-156)Online publication date: 7-Feb-2025
https://doi.org/10.4018/979-8-3693-4219-0.ch003
Van Der Veer GHumayoun SMemmesheimer VEbert A(2024)Experience 2.0 and Beyond - Engineering Cross Devices and Multiple RealitiesCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3662838(108-110)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3662838
Kulkarni SKulkarni A(2024)Review on use of Immersive Technologies in Tourism2024 International Conference on Intelligent Computing and Emerging Communication Technologies (ICEC)10.1109/ICEC59683.2024.10837334(1-7)Online publication date: 23-Nov-2024
https://doi.org/10.1109/ICEC59683.2024.10837334
Show More Cited By

Index Terms

Immersive mixed reality object interaction for collaborative context-aware mobile training and exploration
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
    2. Interaction techniques
2. Information systems
  1. Information systems applications
    1. Mobile information processing systems
    2. Spatial-temporal systems
      1. Location based services

Recommendations

Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Exploring User Behaviour in Asymmetric Collaborative Mixed Reality
VRST '22: Proceedings of the 28th ACM Symposium on Virtual Reality Software and Technology

A common issue for collaborative mixed reality is the asymmetry of interaction with the shared virtual environment. For example, an augmented reality (AR) user might use one type of head-mounted display (HMD) in a physical environment, while a virtual ...
A Framework for Virtual Reality with Tangible Augmented Reality-Based User Interface

In this paper, we propose a framework for virtual reality, I²-NEXT, which enables users to interact with virtual objects by tangible objects in immersive networked virtual environment. The primary goal of this framework is to support rapid development ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MMVE '19: Proceedings of the 11th ACM Workshop on Immersive Mixed and Virtual Environment Systems

June 2019

53 pages

ISBN:9781450362993

DOI:10.1145/3304113

Conference Chairs:
Mario Montagud
i2CAT Foundation & University of Valencia, Spain
,
Francesca De Simone
CWI, Netherlands
,
General Chair:
Niall Murray
Athlone Institute of Technology, Ireland

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MMSys '19

Sponsor:

SIGMM

MMSys '19: 10th ACM Multimedia Systems Conference

June 18, 2019

Massachusetts, Amherst

Acceptance Rates

MMVE '19 Paper Acceptance Rate 9 of 18 submissions, 50%;

Overall Acceptance Rate 26 of 44 submissions, 59%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
262
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)3

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Damaševičius RSidekerskienė T(2025)Technology in Digital Escape RoomsDigital Escape Room Designs in Education10.4018/979-8-3693-4219-0.ch003(107-156)Online publication date: 7-Feb-2025
https://doi.org/10.4018/979-8-3693-4219-0.ch003
Van Der Veer GHumayoun SMemmesheimer VEbert A(2024)Experience 2.0 and Beyond - Engineering Cross Devices and Multiple RealitiesCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3662838(108-110)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3662838
Kulkarni SKulkarni A(2024)Review on use of Immersive Technologies in Tourism2024 International Conference on Intelligent Computing and Emerging Communication Technologies (ICEC)10.1109/ICEC59683.2024.10837334(1-7)Online publication date: 23-Nov-2024
https://doi.org/10.1109/ICEC59683.2024.10837334
Memmesheimer VKlingshirn KRavani BEbert A(2023)Move’n’Hold: Scalable Device-Based Interaction for Mixed Reality Handheld DisplaysProceedings of the European Conference on Cognitive Ergonomics 202310.1145/3605655.3605656(1-8)Online publication date: 19-Sep-2023
https://dl.acm.org/doi/10.1145/3605655.3605656
Memmesheimer VEbert A(2022)Scalable Extended Reality: A Future Research AgendaBig Data and Cognitive Computing10.3390/bdcc60100126:1(12)Online publication date: 26-Jan-2022
https://doi.org/10.3390/bdcc6010012
Kyian STeather R(2021)Selection Performance Using a Smartphone in VR with Redirected InputProceedings of the 2021 ACM Symposium on Spatial User Interaction10.1145/3485279.3485292(1-12)Online publication date: 9-Nov-2021
https://dl.acm.org/doi/10.1145/3485279.3485292
Botev J(2020)Self-Integration in Mediated-Reality Systems: a Socio-Technical Perspective2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)10.1109/ACSOS-C51401.2020.00030(60-61)Online publication date: Aug-2020
https://doi.org/10.1109/ACSOS-C51401.2020.00030

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten