skip to main content
10.1145/2331714.2331718acmotherconferencesArticle/Chapter ViewAbstractPublication PagesvricConference Proceedingsconference-collections
research-article

Embodied interaction with complex neuronal data in mixed-reality

Published: 28 March 2012 Publication History

Abstract

The study of natural and artificial phenomena generates massive amounts of data in many areas of research. This data is frequently left unused due to the lack of tools to effectively extract, analyze and understand it. Visual representation techniques can play a key role in helping to discover patterns and meaning within this data. Neuroscience is one of the scientific fields that generates the most extensive datasets. For this reason we built a 3D real-time visualization system to graphically represent the massive connectivity of neuronal network models in the eXperience Induction Machine (XIM). The XIM is an immersive space equipped with a number of sensors and effectors that we constructed to conduct experiments in mixed-reality. Using this infrastructure we developed an embodied interaction framework that allows the user to move freely in the space and navigate through the neuronal system. We conducted an empirical evaluation of the impact of different navigation mappings on the understanding of a neuronal dataset. Our results revealed that different navigation mappings affect the structural understanding of the system and the involvement with the data presented.

References

[1]
D. Belcher, M. Billinghurst, S. E. Hayes, and R. Stiles. Using augmented reality for visualizing complex graphs in three dimensions. The Second IEEE and ACM International Symposium on Mixed and Augmented Reality 2003 Proceedings, pages 84--93, 2006.
[2]
G. Bell, T. Hey, and A. Szalay. Beyond the Data Deluge. Science, 323(5919):1297--1298, 2009.
[3]
U. Bernardet, S. Bermúdez i Badia, A. Duff, M. Inderbitzin, S. L. Groux, J. Manzolli, Z. Mathews, A. Mura, A. Väljamäe, and P. F. M. J. Verschure. The eXperience Induction Machine: A New Paradigm for Mixed Reality Interaction Design and Psychological Experimentation. In E. Dubois, P. Gray, and L. Nigay, editors, The Engineering of Mixed Reality Systems. Springer, 1 edition, Nov. 2009.
[4]
U. Bernardet, S. Bermúdez i Badia, and P. F. M. J. Verschure. The eXperience Induction Machine and its Role in the Research on Presence. In Proceedings of the 10th annual international workshop on Presence, number Xim, pages 1--7, 2007.
[5]
U. Bernardet, M. Inderbitzin, S. Wierenga, A. Väljamäe, A. Mura, and P. F. M. J. Verschure. Validating presence by relying on recollection: Human experience and performance in the mixed reality system XIM. In L. G. Anna Spagnolli, editor, 11th Annual International Workshop on Presence Padova Italy, volume 54, pages 16--18. CLEUP Cooperativa Libraria Universitaria Padova, 2008.
[6]
U. Bernardet, A. Väljamäe, M. Inderbitzin, S. Wierenga, A. Mura, and P. F. M. J. Verschure. Quantifying human subjective experience and social interaction using the eXperience Induction Machine. Brain research bulletin, 85:305--312, Nov. 2010.
[7]
U. Bernardet and P. F. M. J. Verschure. iqr: a tool for the construction of multi-level simulations of brain and behaviour. Neuroinformatics, 8(2):113--134, 2010.
[8]
C. J. Bohil, B. Alicea, and F. a. Biocca. Virtual reality in neuroscience research and therapy. Nature Reviews Neuroscience, 12(December), Nov. 2011.
[9]
S. K. Card, J. D. Mackinlay, and B. Shneiderman. Readings in information visualization. The Morgan Kaufmann series in interactive technologies H1 - UB Hildesheim H2 - CSC 732: C07, b TS - Gemeinsamer Bibliotheksverbund (GBV). Morgan Kaufmann, 1999.
[10]
R. H. Creighton. Unity 3D Game Development by Example. Packt Publishing, 2010.
[11]
T. Delbruck, A. Whatley, R. Douglas, K. Eng, K. Hepp, and P. F. M. J. Verschure. A tactile luminous floor for an interactive autonomous space. Robotics and Autonomous Systems, 55(6):433--443, 2007.
[12]
J. V. Ernst and A. C. Clark. Technology-Based Content through Virtual and Physical Modeling: A National Research Study. Journal of Technology Education, 20(2):23--36, 2009.
[13]
J. Gammack and C. Hodkinson. Virtual reality, involvement and the consumer interface. Journal of Organizational and End User Computing JOEUC, 15(4):78--96, 2003.
[14]
S. Gerhard, A. Daducci, A. Lemkaddem, R. Meuli, J.-P. Thiran, and P. Hagmann. The Connectome Viewer Toolkit: An Open Source Framework to Manage, Analyze, and Visualize Connectomes. Frontiers in neuroinformatics, 5(June):15, 2011.
[15]
I. Gorton, P. Greenfield, A. Szalay, and R. Williams. Data-Intensive Computing in the 21st Century. Computer, 41:30--32, 2008.
[16]
R. Grush. The Emulation Theory of Representation: Motor Control, Imagery, and Perception. Behavioral and Brain Sciences, 27(03):2010--2010, 2004.
[17]
J. Gutierrez-Maldonado, O. Gutierrez-Martinez, D. Loreto, C. Peñaloza, and R. Nieto. Presence, involvement and efficacy of a virtual reality intervention on pain. Studies In Health Technology And Informatics, 154:97--101, 2010.
[18]
P. Hagmann, L. Cammoun, X. Gigandet, R. Meuli, C. J. Honey, V. J. Wedeen, and O. Sporns. Mapping the Structural Core of Human Cerebral Cortex. PLoS Biology, 6(7):15, 2008.
[19]
W.-K. Jeong, J. Beyer, M. Hadwiger, A. Vazquez, H. Pfister, and R. T. Whitaker. Scalable and interactive segmentation and visualization of neural processes in EM datasets. IEEE Transactions on Visualization and Computer Graphics, 15(6):1505--1514, 2009.
[20]
S. Le Groux, J. Manzolli, and P. F. M. J. Verschure. Interactive Sonification of the Spatial Behavior of Human and Synthetic Characters in a Mixed-Reality Environment. In Proceedings of the 10th Annual International Workshop on Presence, pages 27--34, 2007.
[21]
J. Leigh, C. A. Vasilakis, T. A. Defanti, R. Grossman, C. Assad, B. Rasnow, A. Protopappas, E. De Schutter, and J. M. Bower. Virtual reality in computational neuroscience. In Virtual Reality Applications, pages 293--306. Academic Press, 1995.
[22]
Z. Mathews, S. Bermúdez i Badia, and P. F. M. J. Verschure. A Novel Brain-Based Approach for Multi-Modal Multi-Target Tracking in a Mixed Reality Space. In Proceedings of 4th INTUITION International Conference and Workshop on Virtual Reality, number 1, Athens, Greece, 2007.
[23]
S. A. Mednick. The associative basis of the creative process. Psychological Review, pages 220--232, 1962.
[24]
T. Metzinger. Empirical perspectives from the self-model theory of subjectivity: a brief summary with examples. Brain, 168(07):215--245, 2008.
[25]
J. K. O'Regan and A. Noë. A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5):939--73; discussion 973--1031, 2001.
[26]
B. E. Riecke, B. Bodenheimer, T. P. McNamara, B. Williams, P. Peng, and D. Feuereissen. Do we need to walk for effective virtual reality navigation? physical rotations alone may suffice. In C. Hölscher, T. F. Shipley, M. O. Belardinelli, J. A. Bateman, and N. Newcombe, editors, Proceedings of Spatial Cognition2010, volume 6222 of Lecture Notes in Computer Science, pages 234--247. Springer-Verlag, 2010.
[27]
T. Schubert, F. Friedmann, and H. Regenbrecht. Embodied presence in virtual environments. Visual representations and interpretations, 20(5687):269--278, 1999.
[28]
O. Sporns, G. Tononi, and R. Kötter. The Human Connectome: A Structural Description of the Human Brain. PLoS Comput Biol, 1(4):e42, 2005.
[29]
M. J. Tarr and W. H. Warren. Virtual reality in behavioral neuroscience and beyond. Nature Neuroscience, 5:1089--1092, 2002.
[30]
D. M. von Kapri Anette, Potjans Tobias C., Kuhlen Torsten. A virtual reality exploration tool for multi-scale data from brain-scale simulations. Frontiers in Neuroinformatics, 2011.
[31]
S. Wang, Z. Mao, C. Zeng, H. Gong, S. Li, and B. Chen. A New Method of Virtual Reality Based on Unity3D. Brain, pages 1--5, 2010.

Cited By

View all
  • (2021)Show Me How You Interact, I Will Tell You What You Think: Exploring the Effect of the Interaction Style on Users’ Sensemaking about Correlation and Causation in DataProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462083(564-575)Online publication date: 28-Jun-2021
  • (2021)Future memory: a digital humanities approach for the preservation and presentation of the history of the Holocaust and Nazi crimesHolocaust Studies10.1080/17504902.2021.197917828:3(331-357)Online publication date: 14-Dec-2021
  • (2019)Collaborative Visual Analysis with Multi-level Information Sharing Using a Wall-Size Display and See-Through HMDs2019 IEEE Pacific Visualization Symposium (PacificVis)10.1109/PacificVis.2019.00010(11-20)Online publication date: Apr-2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
VRIC '12: Proceedings of the 2012 Virtual Reality International Conference
March 2012
77 pages
ISBN:9781450312431
DOI:10.1145/2331714
  • Conference Chair:
  • Simon RICHIR
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 March 2012

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. XIM
  2. complex datasets
  3. embodied interaction
  4. iqr
  5. mixed-reality
  6. navigation
  7. neuronal networks
  8. visualization

Qualifiers

  • Research-article

Funding Sources

Conference

VRIC '12

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)14
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2021)Show Me How You Interact, I Will Tell You What You Think: Exploring the Effect of the Interaction Style on Users’ Sensemaking about Correlation and Causation in DataProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462083(564-575)Online publication date: 28-Jun-2021
  • (2021)Future memory: a digital humanities approach for the preservation and presentation of the history of the Holocaust and Nazi crimesHolocaust Studies10.1080/17504902.2021.197917828:3(331-357)Online publication date: 14-Dec-2021
  • (2019)Collaborative Visual Analysis with Multi-level Information Sharing Using a Wall-Size Display and See-Through HMDs2019 IEEE Pacific Visualization Symposium (PacificVis)10.1109/PacificVis.2019.00010(11-20)Online publication date: Apr-2019
  • (2018)Clusters, Trends, and OutliersProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173664(1-12)Online publication date: 21-Apr-2018
  • (2017)Investigating the Use of Spatial Interaction for 3D Data Visualization on Mobile DevicesProceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces10.1145/3132272.3134125(62-71)Online publication date: 17-Oct-2017
  • (2015)Collaborative magic lens graph explorationSIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications10.1145/2818427.2818465(1-3)Online publication date: 2-Nov-2015
  • (2015)An Immersive and Interactive Visualization System by Integrating Distinct PlatformsProceedings of the 2015 19th International Conference on Information Visualisation10.1109/iV.2015.74(403-410)Online publication date: 22-Jul-2015
  • (2014)Subliminal response priming in mixed realityPresence: Teleoperators and Virtual Environments10.1162/PRES_a_0017123:1(1-17)Online publication date: 1-Feb-2014
  • (2014)BrainX3Proceedings of the 2014 Virtual Reality International Conference10.1145/2617841.2620726(1-4)Online publication date: 9-Apr-2014
  • (2014)Manipulating complex network structures in virtual reality and 3D printing of the resultsProceedings of the 2014 Virtual Reality International Conference10.1145/2617841.2620718(1-2)Online publication date: 9-Apr-2014
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media