Kevin Fan
Susumu Tachi
ABSTRACT
We present Reality Jockey, a system that confuses the participant's perception of the reality by mixing in a recorded past-reality. The participant will be immersed in a spatialized 3D sound environment that is a mix of sounds from the reality and from the past. The sound environment from the past is augmented with haptic feedback in cross-modality. The haptic feedback is associated with certain sounds such as the vibration in the table when stuff is placed on the table to make the illusion of it happening in live. The seamless transition between live and past creates immersive experience of past events. The blending of live and past allows interactivity. To validate our system, we conducted user studies on 1) does blending live sensations improve such experiences, and 2) how beneficial is it to provide haptic feedbacks in recorded pasts. Potential applications are suggested to illustrate the significance of Reality Jockey.
Supplemental Material
- Begault, D. 3-D Sound for Virtual Reality and Multimedia. AP Professional, 1994. Google Scholar
Digital Library
- Botvinick, M., and Cohen, J. Rubber hands "feel" touch that eyes see. Nature 391(6669):756.Google Scholar
- Chalmers, A., Howard, D., and Moir, C. Real Virtuality: A step change from Virtual Reality. In Proc. SCCG '09, Stephen N. Spencer (Ed.). ACM (2009), 9--16. Google ScholarDigital Library
- Colby, K. M., Hilf, F. D., Weber, S., and Kraemer, H. Turing-like Indistinguishability Tests for the Validation of a Computer Simulation of Paranoid Processes. Artificial Intelligence 3: 199--221.Google Scholar
- Csikszentmihalyi, M. Flow: the Psychology of Optimal Experience. Harper and Row, 1990.Google Scholar
- FMOD. http://www.fmod.org/Google Scholar
- Fuji Television, Time Trip View. http://blog.fujitv.co.jp/airmail/E20120627002.htmlGoogle Scholar
- Iwai, T. Another Time, Another Space. http://www.leonardo.info/gallery/gallery332/iwai.htmlGoogle Scholar
- Larsson, P., Västfjäll, D., and Kleiner, M. Perception of self-motion and presence in auditory virtual environments. In Proc. Seventh Annual International Workshop Presence (2004), 252--258.Google Scholar
- Law, A. W., Peck, B., Visell, Y., Kry, P., and Cooperstock, J. 2008. A multi-modal floor-space for experiencing material deformation underfoot in virtual reality. In Proc. IEEE HAVE'08, 126--131.Google Scholar
- MacDonald, J. A Conversation With My 12 Year Old Self. http://youtu.be/XFGAQrEUaeUGoogle Scholar
- Minamizawa, K., Kakehi, Y., Nakatani, M., Mihara, S., and Tachi, S. TECHTILE toolkit: a prototyping tool for design and education of haptic media. In Proc. VRIC '12, ACM (2012), 2 pages. Google ScholarDigital Library
- MIRAGE: Performance Art with Substitutional Reality system. http://mirage.grinder-man.com/ (in Japanese)Google Scholar
- Naef, M., Staadt, O., and Gross, M. Spatialized audio rendering for immersive virtual environments. In Proc. VRST '02, ACM, pp. 65--72. Google ScholarDigital Library
- Riecke, B. E., Feuereissen, D., and Rieser, J. J. Auditory self-motion illusions ("circular vection") can be facilitated by vibrations and the potential for actual motion. In Proc. APGV '08. Google ScholarDigital Library
- Roediger, H. L., and McDermott, K. B. Creating False Memories: Remembering Words Not Presented in Lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, Vol 21(4), 1995, 803--814.Google Scholar
- Suzuki, K., Wakisaka, S. and Fujii, N. Substitutional Reality System: A Novel Experimental Platform for Experiencing Alternative Reality. Sci. Rep. 2, 459.Google Scholar
- Tachi, S. Telexistence. World Scientific, 2009. Google ScholarDigital Library
- Takeuchi, T., Nakashima, T., Nishimura, K., and Hirose, M. PRIMA: Parallel Reality-based Interactive Motion Area. In ACM SIGGRAPH 2011 Posters (SIGGRAPH '11). ACM (2011), 1 page. Google ScholarDigital Library
- Tamaki, E., Miyaki, T., and Rekimoto, J. PossessedHand: a hand gesture manipulation system using electrical stimuli. In Proc. AH '10. ACM (2010), Article 2 , 5 pages. Google ScholarDigital Library
- Turing, A. M. Computing Machinery and Intelligence. Mind LIX (236): 433--460.Google Scholar
- Weizenbaum, J. ELIZA - A Computer Program For the Study of Natural Language Communication Between Man And Machine. Communications of the ACM 9 (1): 36--45. Google ScholarDigital Library
Index Terms
- Reality jockey: lifting the barrier between alternate realities through audio and haptic feedback
Recommendations
Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsExperiencing Virtual Reality in domestic and other uncontrolled settings is challenging due to the presence of physical objects and furniture that are not usually defined in the Virtual Environment. To address this challenge, we explore the concept of ...
atmoSphere: designing cross-modal music experiences using spatial audio with haptic feedback
SIGGRAPH '17: ACM SIGGRAPH 2017 Emerging TechnologiesWe use cross-Modal correspondence -the interaction between two or more sensory modalities- to create an engaging user experience. We present atmoSphere, a system that provides users immersive music experiences using spatial audio and haptic feedback. We ...
Journal of My Journey: Seamless Interaction in Virtuality and Reality with Digital Fabrication and Sensory Feedback
SIGGRAPH '22: ACM SIGGRAPH 2022 Immersive PavilionWhether in virtuality or reality, the decisions or actions pursue the goal we acquire in the virtual or real world. Besides, the decision or action that the users make in virtuality or reality only affects one physical or virtual environment. Previous ...
Comments