skip to main content
10.1145/2470654.2481353acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Reality jockey: lifting the barrier between alternate realities through audio and haptic feedback

Published: 27 April 2013 Publication History

Abstract

We present Reality Jockey, a system that confuses the participant's perception of the reality by mixing in a recorded past-reality. The participant will be immersed in a spatialized 3D sound environment that is a mix of sounds from the reality and from the past. The sound environment from the past is augmented with haptic feedback in cross-modality. The haptic feedback is associated with certain sounds such as the vibration in the table when stuff is placed on the table to make the illusion of it happening in live. The seamless transition between live and past creates immersive experience of past events. The blending of live and past allows interactivity. To validate our system, we conducted user studies on 1) does blending live sensations improve such experiences, and 2) how beneficial is it to provide haptic feedbacks in recorded pasts. Potential applications are suggested to illustrate the significance of Reality Jockey.

Supplementary Material

suppl.mov (chi1330-file3.m4v)
Supplemental video

References

[1]
Begault, D. 3-D Sound for Virtual Reality and Multimedia. AP Professional, 1994.
[2]
Botvinick, M., and Cohen, J. Rubber hands "feel" touch that eyes see. Nature 391(6669):756.
[3]
Chalmers, A., Howard, D., and Moir, C. Real Virtuality: A step change from Virtual Reality. In Proc. SCCG '09, Stephen N. Spencer (Ed.). ACM (2009), 9--16.
[4]
Colby, K. M., Hilf, F. D., Weber, S., and Kraemer, H. Turing-like Indistinguishability Tests for the Validation of a Computer Simulation of Paranoid Processes. Artificial Intelligence 3: 199--221.
[5]
Csikszentmihalyi, M. Flow: the Psychology of Optimal Experience. Harper and Row, 1990.
[6]
FMOD. http://www.fmod.org/
[7]
Fuji Television, Time Trip View. http://blog.fujitv.co.jp/airmail/E20120627002.html
[8]
Iwai, T. Another Time, Another Space. http://www.leonardo.info/gallery/gallery332/iwai.html
[9]
Larsson, P., Västfjäll, D., and Kleiner, M. Perception of self-motion and presence in auditory virtual environments. In Proc. Seventh Annual International Workshop Presence (2004), 252--258.
[10]
Law, A. W., Peck, B., Visell, Y., Kry, P., and Cooperstock, J. 2008. A multi-modal floor-space for experiencing material deformation underfoot in virtual reality. In Proc. IEEE HAVE'08, 126--131.
[11]
MacDonald, J. A Conversation With My 12 Year Old Self. http://youtu.be/XFGAQrEUaeU
[12]
Minamizawa, K., Kakehi, Y., Nakatani, M., Mihara, S., and Tachi, S. TECHTILE toolkit: a prototyping tool for design and education of haptic media. In Proc. VRIC '12, ACM (2012), 2 pages.
[13]
MIRAGE: Performance Art with Substitutional Reality system. http://mirage.grinder-man.com/ (in Japanese)
[14]
Naef, M., Staadt, O., and Gross, M. Spatialized audio rendering for immersive virtual environments. In Proc. VRST '02, ACM, pp. 65--72.
[15]
Riecke, B. E., Feuereissen, D., and Rieser, J. J. Auditory self-motion illusions ("circular vection") can be facilitated by vibrations and the potential for actual motion. In Proc. APGV '08.
[16]
Roediger, H. L., and McDermott, K. B. Creating False Memories: Remembering Words Not Presented in Lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, Vol 21(4), 1995, 803--814.
[17]
Suzuki, K., Wakisaka, S. and Fujii, N. Substitutional Reality System: A Novel Experimental Platform for Experiencing Alternative Reality. Sci. Rep. 2, 459.
[18]
Tachi, S. Telexistence. World Scientific, 2009.
[19]
Takeuchi, T., Nakashima, T., Nishimura, K., and Hirose, M. PRIMA: Parallel Reality-based Interactive Motion Area. In ACM SIGGRAPH 2011 Posters (SIGGRAPH '11). ACM (2011), 1 page.
[20]
Tamaki, E., Miyaki, T., and Rekimoto, J. PossessedHand: a hand gesture manipulation system using electrical stimuli. In Proc. AH '10. ACM (2010), Article 2, 5 pages.
[21]
Turing, A. M. Computing Machinery and Intelligence. Mind LIX (236): 433--460.
[22]
Weizenbaum, J. ELIZA - A Computer Program For the Study of Natural Language Communication Between Man And Machine. Communications of the ACM 9 (1): 36--45.

Cited By

View all
  • (2022)Cross-modal associations between paintings and sounds: Effects of embodimentPerception10.1177/0301006622112645251:12(871-888)Online publication date: 11-Oct-2022
  • (2017)atmoSphereACM SIGGRAPH 2017 Emerging Technologies10.1145/3084822.3084845(1-2)Online publication date: 30-Jul-2017
  • (2017)ACM SIGGRAPH 2017 Emerging TechnologiesundefinedOnline publication date: 30-Jul-2017
  • Show More Cited By

Index Terms

  1. Reality jockey: lifting the barrier between alternate realities through audio and haptic feedback

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
    April 2013
    3550 pages
    ISBN:9781450318990
    DOI:10.1145/2470654
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 April 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. cross-modality
    2. haptic
    3. illusion
    4. spatial sound
    5. substitutional reality

    Qualifiers

    • Research-article

    Conference

    CHI '13
    Sponsor:

    Acceptance Rates

    CHI '13 Paper Acceptance Rate 392 of 1,963 submissions, 20%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)29
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 23 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Cross-modal associations between paintings and sounds: Effects of embodimentPerception10.1177/0301006622112645251:12(871-888)Online publication date: 11-Oct-2022
    • (2017)atmoSphereACM SIGGRAPH 2017 Emerging Technologies10.1145/3084822.3084845(1-2)Online publication date: 30-Jul-2017
    • (2017)ACM SIGGRAPH 2017 Emerging TechnologiesundefinedOnline publication date: 30-Jul-2017
    • (2014)Workshop on assistive augmentationCHI '14 Extended Abstracts on Human Factors in Computing Systems10.1145/2559206.2560473(103-106)Online publication date: 26-Apr-2014

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media