skip to main content
10.1145/2642918.2647383acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units

Published: 05 October 2014 Publication History

Abstract

RoomAlive is a proof-of-concept prototype that transforms any room into an immersive, augmented entertainment experience. Our system enables new interactive projection mapping experiences that dynamically adapts content to any room. Users can touch, shoot, stomp, dodge and steer projected content that seamlessly co-exists with their existing physical environment. The basic building blocks of RoomAlive are projector-depth camera units, which can be combined through a scalable, distributed framework. The projector-depth camera units are individually auto-calibrating, self-localizing, and create a unified model of the room with no user intervention. We investigate the design space of gaming experiences that are possible with RoomAlive and explore methods for dynamically mapping content based on room layout and user position. Finally we showcase four experience prototypes that demonstrate the novel interactive experiences that are possible with RoomAlive and discuss the design challenges of adapting any game to any room.

Supplementary Material

suppl.mov (uistf2943-file3.mp4)
Supplemental video

References

[1]
Batlle, J., Mouaddib, E., and Salvi, J. Recent progress in coded structured light as a technique to solve correspondence problem: A survey. Pattern Recognition 31, 7 (1998), 963--982.
[2]
Behringer, R. Placing Artificial Objects in Real Scenes. Proc. of IWAR, (1998).
[3]
Benko, H., Jota, R., and Wilson, A. MirageTable : Freehand Interaction on a Projected Augmented Reality Tabletop. ACM CHI, (2012).
[4]
Bimber, O. and Raskar, R. Spatial augmented reality: Merging real and virtual worlds. AK Peters Ltd, 2005.
[5]
Bimber, O. PlayReal: Spatial Augmented Reality Games. 2008.
[6]
Brown, M., Majumder, A., and Yang, R. Camerabased calibration techniques for seamless multiprojector displays. Proc. of Visualization and Computer Graphcis, (2005), 193--206.
[7]
Brox, T., Papenberg, N., and Weickert, J. High Accuracy Optical Flow Estimation Based on a Theory for Warping. Proc. of ECCV, (2004), 25--36.
[8]
Butler, A., Izadi, S., Hilliges, O., Molyneaux, D., Hodges, S., and Kim, D. Shake 'n' Sense : Reducing Interference for Overlapping Structured Light Depth Cameras. ACM CHI, (2012), 1933-- 1936.
[9]
Cruz-Neira, C., Sandin, D., and DeFanti, T. Surround-screen projection-based virtual reality: the design and implementation of the CAVE. ACM SIGGRAPH, (1993).
[10]
Fuchs, H. and Frahm, J.-M. Scanning and tracking dynamic objects with commodity depth cameras. Proc. of ISMAR, Ieee (2013), 99--106. Figure 15. Calibration errors between units result in ghosting artifacts in projector overlapping regions. (left) The character in a single projector, and (right) in an overlapping region.
[11]
Hancock, M., Nacenta, M., Gutwin, C., and Carpendale, S. The effects of changing projection geometry on the interpretation of 3D orientation on tabletops. Proc. of ITS, (2009), 175--182.
[12]
Harrison, C., Benko, H., Wilson, A.D., and Way, O.M. OmniTouch : Wearable multitouch interaction everywhere. Proc. UIST, (2011), 441--450.
[13]
Hilliges, O., Kim, D., Izadi, S., Weiss, M., and Wilson, A. HoloDesk. Proc. of CHI, ACM Press (2012), 2421--2432.
[14]
Hou, J., Nam, Y., Peng, W., and Lee, K.M. Effects of screen size, viewing angle, and players' immersion tendencies on game experience. Computers in Human Behavior 28, 2 (2012), 617-- 623.
[15]
Illingworth, J. and Kittler, J. A survey of the Hough transform. Computer Vision, Graphics, and Image Processing 1, 44 (1988).
[16]
Jones, B., Benko, H., Ofek, E., and Wilson, A.D. IllumiRoom: Peripheral Projected Illusions for Interactive Experiences. ACM CHI, (2013).
[17]
Jones, B., Sodhi, R., Campbell, R., Garnett, G., and Bailey, B.P. Build Your World and Play In It: Interacting with Surface Particles on Complex Objects. IEEE ISMAR, (2010).
[18]
Kainz, B., Hauswiesner, S., Reitmayr, G., et al. OmniKinect : Real-Time Dense Volumetric Data Acquisition and Applications. Proc. of VR, (2012), 25--32.
[19]
Lee, J.C., Dietz, P.H., Maynes-Aminzade, D., Raskar, R., and Hudson, S.E. Automatic projector calibration with embedded light sensors. UIST, ACM (2004), 123--126.
[20]
Mine, M.R., van Baar, J., Grundhofer, A., Rose, D., and Yang, B. Projection-Based AR in Disney Theme Parks. Computer 45, 7 (2012), 32--40.
[21]
Pinhanez, C. The Everywhere Displays Projector: A Device to create ubiquitous graphical interfaces. Proc. UbiComp, (2001), 315--331.
[22]
ProCams. http://www.procams.org. 2007.
[23]
Projection Mapping Central. Projection Mapping Central. http://projection-mapping.org, 2014.
[24]
R. Sekular, R.B. Perception 2nd Edition. (1990).
[25]
Raskar, R., van Baar, J., Beardsley, P., Willwacher, T., Rao, S., and Forlines, C. iLamps: geometrically aware and self-configuring projectors. ACM TOG 223, (2003), 809--818.
[26]
Raskar, R. and Beardsley, P. A self-correcting projector. Proc. of CVPR 2001, IEEE Comput. Soc (2001), 504--508.
[27]
Raskar, R., Brown, M.S., Yang, R., et al. Multiprojector displays using camera-based registration. IEEE Visualization, (1999), 161--168.
[28]
Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., and Fuchs, H. The office of the future: A unified approach to image-based modeling and spatially immersive displays. Proc. SIGGRAPH, ACM (1998), 179--188.
[29]
Raskar, R., Welch, G., and Fuchs, H. Spatially Augmented Reality. (1998), 1--7.
[30]
Raskar, R., Welch, G., Low, K.L., and Bandyopadhyay, D. Shader lamps: Animating real objects with image-based illumination. Proc. Eurographics, Springer Verlag Wien (2001), 89-- 100.
[31]
Sajadi, B., Member, S., and Majumder, A. AutoCalibration of Multi-Projector CAVE-like Immersive Environments. Trans. on Visualization and Comp. Graphics 17, 1 (2011), 1--13.
[32]
Schulze, P., Acevedo, D., Mangan, J., et al. Democratizing Rendering for Multiple Viewers in Surround VR Systems. IEEE 3DUI, (2012).
[33]
Smith, G., Othenin-girard, A., Whitehead, J., and Wardrip-fruin, N. PCG-Based Game Design : Creating Endless Web. In Proc. of Foundations of Digital Games, (2012).
[34]
Straka, M., Hauswiesner, S., Ruther, M., and Bischof, H. Rapid Skin: Estimating the 3D Human Pose and Shape in Real-Time. Proc of 3DimPVT, Ieee (2012), 41--48.
[35]
VR, O. http://www.oculusvr.com/. .
[36]
Willis, K.D.D. SideBySide : Ad-hoc Multi-user Interaction with Handheld Projectors. ACM UIST, (2011), 431--440.
[37]
Wilson, A., Benko, H., Izadi, S., and Hilliges, O. Steerable augmented reality with the beamatron. Proceedings of the 25th annual ACM symposium on User interface software and technology - UIST '12, ACM Press (2012), 413.
[38]
Wilson, A. Combining Multiple Depth Cameras and Projectors for Interactions On, Above, and Between Surfaces. ACM CHI, (2010).
[39]
Wilson, A.D. PlayAnywhere: a compact interactive tabletop projection-vision system. Proc. of UIST, ACM (2005), 92--102.

Cited By

View all
  • (2024)LuxAR: A Direct Manipulation Projected Display to Extend and Augment Desktop ComputingProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670981(1-12)Online publication date: 3-Jun-2024
  • (2024)Virtual Real Spaces: In-Home Gaming Experiences Bridging Reality and VirtualityCompanion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play10.1145/3665463.3678812(157-162)Online publication date: 14-Oct-2024
  • (2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
  • Show More Cited By

Index Terms

  1. RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology
    October 2014
    722 pages
    ISBN:9781450330695
    DOI:10.1145/2642918
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 05 October 2014

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. projection mapping
    2. projector-camera system
    3. spatial augmented reality

    Qualifiers

    • Research-article

    Conference

    UIST '14

    Acceptance Rates

    UIST '14 Paper Acceptance Rate 74 of 333 submissions, 22%;
    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)221
    • Downloads (Last 6 weeks)12
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)LuxAR: A Direct Manipulation Projected Display to Extend and Augment Desktop ComputingProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670981(1-12)Online publication date: 3-Jun-2024
    • (2024)Virtual Real Spaces: In-Home Gaming Experiences Bridging Reality and VirtualityCompanion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play10.1145/3665463.3678812(157-162)Online publication date: 14-Oct-2024
    • (2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
    • (2024)Welcome to the Jungle: Using ARTV to Balance Media Immersion and Reality Awareness for Passengers on Public TransportProceedings of the 2024 ACM International Conference on Interactive Media Experiences10.1145/3639701.3656321(219-230)Online publication date: 7-Jun-2024
    • (2024)Screen Augmentation Technique Using AR Glasses and Smartphone without External SensorsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3648664(1-5)Online publication date: 11-May-2024
    • (2024)VisTorch: Interacting with Situated Visualizations using Handheld ProjectorsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642857(1-13)Online publication date: 11-May-2024
    • (2024)Jigsaw: Authoring Immersive Storytelling Experiences with Augmented Reality and Internet of ThingsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642744(1-14)Online publication date: 11-May-2024
    • (2024)Watch Your Mouth: Silent Speech Recognition with Depth SensingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642092(1-15)Online publication date: 11-May-2024
    • (2024)Research on 3D reconstruction algorithms for small scenes with weak textureInternational Conference on Algorithm, Imaging Processing, and Machine Vision (AIPMV 2023)10.1117/12.3014370(14)Online publication date: 9-Jan-2024
    • (2024)Projection Mapping under Environmental Lighting by Replacing Room Lights with Heterogeneous ProjectorsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337203130:5(2151-2161)Online publication date: 4-Mar-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media