Abstract
The AR Vis project is a general-purpose interactive data visualization platform for collaborative interaction with scientific data. The platform is designed for augmented reality displays of data supporting multi-user interaction and simulations. Methods developed include a procedural pipeline for data culling, modeling, visualization, and porting to multiuser augmented reality.
A prototype interactive visualization application demonstrates the system via visualization and simulation of magnetic fields. The magnetic field visualizations are attached to physical objects or embedded in the environment. The invisible magnetic fields are transformed into tangible models of nano and geospatial scales magnetic phenomena accessible to a user’s full body (embodied) interaction.
The project seeks to make a significant contribution to scientific visualization. Extending beyond the cognitive impact of traditional scientific visualization, the goal of the AR Vis platform is to additionally leverage human perception and spatial cognition and make data patterns tangible, manipulable and more accessible. In supporting augmented information cognition in scientists and learners, AR Vis design supports data discovery and learning.
The prototype implementation uses physics data modeling of the invisible and largely intangible forces of magnetism across different scales. The project yields both a prototype platform and develops a data visualization pipeline. Both demonstrate a substantial and concrete implementation and demonstration of AR Vis techniques.
Keywords
You have full access to this open access chapter, Download conference paper PDF
1 Introduction
The visualization of scientific data and large network data have been evolving with the arrival of computer graphic techniques. Most recently key developments have involved the ability to process large amounts of data and render them in new and different graphical formats. The array of information remains primarily constrained to 2D dimensional projection (e.g., Tableau) or 2D renderings of 3-dimensional arrays.
Data visualizations are used to access the powers of human cognition. Data visualizations leverage the powerful capacities of human spatial, object, and color perception to detect patterns, trends, and anomalies in physical forms.
The introduction of virtual reality and especially augmented reality has enabled scientists, data analysts, and the general public to interact with scientific and big data visualizations in more intuitive ways. Augmented reality and virtual reality array data in a simulated physical space around the body. In this way, the leverage all of the visualization capabilities of traditional visualization but are further augmented by human spatial cognition and tangible, full body, physically interaction with the data. Virtual and augmented reality array the data around the body and engage in body-centric spatial awareness to detect patterns and comprehend scale and relationships. Full body interaction with data visualizations enables users given them new ways to engage, alter, and “interrogate” the data.
1.1 Visualizing Unseen Phenomena at Various Scales: Magnetic Fields
Magnetic fields are largely invisible phenomena exerting effects in the world. Magnetic fields are pervasive and can be modeled at different scales from the nanoscales to larger geospatial scales such as the interaction of sun and the earth’s magnetic fields. Magnetic fields lend themselves to the possibility of visualization at different scales including the superimposing on the real-world objects, the magnification of smaller scale magnetism, and the visualization of large scale early magnetism at an experiential scale of the body. In the project we start the interactive visualization of Environmentally Mapped View; these interactive visualizations view data mapped directly at one-to-one scale on physical objects or the surrounding environment.
2 Specific Objectives
The specific objectives are:
-
Develop a general-purpose interactive data visualization platform.
-
Visualize, simulate, and overlay natural physical processes on physical objects and natural spaces.
-
Develop and demonstrate the approach via the visualization of magnetic and electromagnetic phenomena.
-
Transform visualization to body-centric, tangible, and interactive scientific visualization
-
Visualize and simulate magnetic fields at different scales.
-
Support improved understanding of physical phenomena for scientific data exploration and education.
3 Methods and Procedures
3.1 Overview
Our development of a graphics and interactive design pipeline for the creation of interactive visualizations of time-based and environmental phenomena rendered in the physical environment to an embedded 3D observer. The procedure for body-centered, tangible experience using a multi-user augmented reality environment we call AR Vis.
3.2 Data Sources
We are building a general-purpose platform and a set of procedures for interactive visualization suited to display spatial and environmental data varying over time and different scales. For the prototype study, we demonstrate the visualization using magnetic data from the Center for Solar-Terrestrial Research at NJIT.
Magnetic Field Data: From the Inviable to the Tangible.
At the beginning of project work, magnetic field data that are used for the study of solar and terrestrial sciences are visualized (Fig. 1).
They are typically acquired by the instrument called “magnetometer” that measures the background (ambient) or time-varying component(s) of magnetic fields. It has a wide range of applications and is one of the widely utilized instruments for geospace research. There are a number of science magnetometers in operation in ground-based observatories and aboard spacecraft.
Magnetic field data sets are publicly available either online (e.g., NASA) or upon request. They are provided in the text-based ASCII format or Common Data Format (CDF) which contains the three-axis components (vector) or scalar (absolute) values to represent magnetic fields around the Earth.
The Earth’s magnetic fields are never constant, being disturbed by various sources. One of them is the continuous flow of charged particles from the Sun called “solar wind.” Thus, monitoring the Earth’s magnetic fields provide critical information about the environment between the Sun and Earth.
A member of this project, Dr. Kim at NJIT Center for Solar-Terrestrial Research (CSTR), is currently in charge of the operation of magnetometers in the polar regions in both hemispheres including Canada, Norway, Greenland, and Antarctica and has extensive experiences in analyzing data from the magnetometers. We are also participating in a satellite program run by NASA.
Here, we visualize such data sets from the various magnetometers and thus to provide the general public with more tangible experience with the unseen forces that surround us.
Data Mining, Reduction, and Culling.
Data mining and culling extract spatial and time-varying components. Data sets are modified and formatted to support 3D visualization tools.
3.3 Visualization and Interactive Simulation Model Building
The magnetic field visualization prototype was created in Unity and the scripts were written in C#. The Microsoft Mixed Reality ToolKit was used to implement the Hololens functionality of the Unity Project.
The Mixed Reality ToolKit was necessary for the project because it allows Unity projects to be directly built onto the Hololens. It also contains scripts and prefabs which allows for a Hololens scene to be setup easily. For example, the Mixed Reality Tool Kit Parent Prefab was used to represent the Hololens’ in the scene. In addition, the prefab contained scripts and other objects attached to it that help with gazing and head position. There are also other prefabs in the Mixed Reality Toolkit that help with input and other essential things needed for a Hololens’ project.
The grid was created using two geometry shaders. The shader language used was Cg, which is a variant of HLSL. A series of vertices are passed to the GPU, the first geometry shader draws a cube at each of these points, while the second geometry shader draws lines connecting neighboring points. In addition, a script calculates the distance between the magnetic object in the scene and the player, as well as the distance between the player and each vertex. The sum of these calculations is adjusted by a manually tweaked value and applied to the vertices’ positions to make them move rapidly or slowly. The number is also used to determine the color of the grid vertices (Fig. 2).
Platform Porting and Interactivity.
The interactive visualizations and simulations are ported for interaction in an NJIT multiuser environment using Hololens and hardware. Hololens is an augmented reality system using optic see-though HMD for blending virtual objects into the physical environment. We begin with the networked dual-use platform (Fig. 3).
Our visualizations Micro Software, Hololens’s application framework. We are able to support multiple spatial computing and visualization experiences to run at the same time.
Multi-observer Environment.
Because users and scientists often interact with data in teams, the interactive simulations design for two or more observers of the environment. Both are able to simultaneously view shared visualization. Models are shared across devices in real time (Fig. 4).
Augmented Reality Views.
The platform yields visualizations at different views using augmented reality. The current prototype supports the visualization of body-centered magnetic fields. Users are instrumented with a magnetometer and move through a real-time dynamic visualization of the magnetic fields in the room or space around their body. Actions include selection or even physical motion modify the data or support interaction.
The platform envisions different types of augmented reality views of magnetic data described below and include an object or environmentally mapped views and observer or “Gods eye” views of very large planetary scale static or real-time magnetic field activity.
4 Future Plans
This project supports the development of a prototype and development pipeline for an augmented reality, visualization interface.
Preliminary user studies examine the usability, interactivity, of the interactive visualization. Learning and discovery outcomes are also be assessed by comparing the spatial augmented reality visualizations to traditional visualizations of the same data.
The multi-user system will be generalizable to the visualization and augmentation of other information including for example architectural and structural visualization, embedded visualization of urban and physical environments, electrical grid and infrastructure visualization, medical patient data visualization, big data visualizations, radiation, and environment mapping.
References
Bhagat, K.K., Liou, W.-K., Michael Spector, J., Chang, C.-Y.: To use augmented reality or not in formative assessment: a comparative study. Interact. Learn. Environ., 1–11 (2018)
Neumann, U., You, S., Hu, J., Jiang, B., Sebe, I.O.: Visualizing reality in an augmented virtual environment. Presence: Teleoperators Virtual Environ. 13(2), 222–233 (2004). https://doi.org/10.1162/1054746041382366
Olshannikova, E., Ometov, A., Koucheryavy, Y., Olsson, T.: Visualizing big data with augmented and virtual reality: challenges and research agenda. J. Big Data 2(1), 22 (2015). https://doi.org/10.1186/s40537-015-0031-2
Patterson, R.E., et al.: A human cognition framework for information visualization. Comput. Graph. 42, 42–58 (2014). https://doi.org/10.1016/j.cag.2014.03.002
Acknowledgments
This project partly founding by the 2018–2019 Faculty Seed grant from NJIT.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Kum-Biocca, H.H., Kim, H., Biocca, F., Cho, Y. (2019). AR-VIS: Augmented Reality Interactive Visualization Environment for Exploring Dynamic Scientific Data. In: Stephanidis, C., Antona, M. (eds) HCI International 2019 – Late Breaking Posters. HCII 2019. Communications in Computer and Information Science, vol 1088. Springer, Cham. https://doi.org/10.1007/978-3-030-30712-7_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-30712-7_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30711-0
Online ISBN: 978-3-030-30712-7
eBook Packages: Computer ScienceComputer Science (R0)