Keywords

1 Introduction

When viewing images in film, television, mobile game consoles, etc., symptoms of motion sickness such as eye fatigue, nausea, and dizziness may occur. Though the mechanism for symptoms of such visually induced motion sickness is unknown, shaking images and recognizing rotation are said to be the causes. In recent years, it is possible to view three-dimensional (3D) images not only in 3D films at movie theaters and attractions in amusement parks, but also at home through 3D television, mobile game consoles, and head mounted displays. Compared to the past, when 3D images could only be viewed in limited locations, the technology has become more common, and opportunities to view 3D images have increased. 3D sickness is being reported as 3D images that enhance virtual reality have become more prevalent. Investigating causes and countermeasures is increasingly urgent from a public health point of view.

Sensory conflict theory is a well-known explanation for the cause of visually induced motion sickness. The theory describes a discrepancy between visual cues and cognition of vestibular/somatosensory sensations that triggers sickness [1] (as when visual information indicates motion while the body is actually stationary. There are other theories such as vergence-accommodation mismatch and poison theory, but these fail to elucidate the cause in sufficient detail. At present, studies on the impact 3D images have on the body are insufficient. Therefore, further accumulation of empirical studies is essential.

It has been confirmed in a stabilometry that compared to tracking a target while viewing 3D images, using peripheral vision for the entire screen triggers 3D sickness more readily [2]. Therefore, in this study, to elucidate the cause for 3D sickness, we examined the impact of viewing 3D images on brain activities and used functional near-infrared spectroscopy (fNIRS) to measure potential changes in brain activities during tracking and using peripheral vision for comparison. Also, we measured eye movements to confirm whether subjects were tracking or using peripheral vision while viewing images.

2 fNIRS

fNIRS uses a property of hemoglobin in which near-infrared light is absorbed, and can measure the rate of blood flow in a body noninvasively. When fNIRS is applied to the brain, it is able to measure changes in cerebral blood from within 2–3 cm from the scalp and identify activated sites. In other words, fNIRS is a test that can capture temporal changes in activation reaction of cerebral cortex accompanied with biological activities in a comprehensive and noninvasive manner [3,4,5,6,7,8,9].

Visible light with a wavelength of 400–700 nm has a high absorption of hemoglobin and other biological constituents, and since wavelength longer than near-infrared light absorbs more water, infrared light cannot move directly through the body. However, the absorption rate of hemoglobin in near-infrared light with a wavelength of 700–900 nm is higher than that of water. In this range of wavelength, extinction coefficients of oxygenated hemoglobin and deoxygenated hemoglobin are different, and the isosbestic point is near 805 nm. fNIRS uses multiple wavelengths for spectroscopic measurement and calculates concentrations of oxygenated hemoglobin and deoxygenated hemoglobin in blood [mg/L], and the total hemoglobin, which is the sum of the two [8].

3 Experimental Methods

3.1 fNIRS Measurements

We performed our experiment on 14 healthy males and females whose age ranged between 20 to 26 (mean ± standard deviation: 21.0 ± 1.8). The 3D image used for the experiment was recreated based on Sky Crystal (Olympus Memory Works Corp, Tokyo) with permission (Fig. 1). For fNIRS measurements, we used LABNIRS (Shimadzu Corporation, Kyoto). Images were displayed using 3D Display 55UF8500 (LG, Seoul). Measurements were taken in a darkroom, and subjects were seated for the experiment (Fig. 2). Subjects wore a head holder with transmission and reception probes. We used ch 1–ch 48 so that channels were arranged across the entire cerebral surface (Fig. 3). Following the 70-s test with eyes closed (Pre), subjects tracked 3D images for 70 s and used peripheral vision for another 70 s. These series were continued in five sets, and changes in cerebral blood flow were measured at 55 Hz. Two types of 3D images (with and without background) were used. A low-pass filter of 0.15 Hz was applied to the concentration of oxygenated hemoglobin obtained in the experiment to smooth the high-frequency component to calculate the integral value. Integral values of oxygenated hemoglobin concentration for Pre, tracking, and peripheral vision for each channel were compared using Wilcoxon signed-rank test. The significance level is 0.05.

Fig. 1.
figure 1

Visual Stimulus; an image with background (a), an image without background (b)

Full size image
Fig. 2.
figure 2

The visual distance was set to be 1.86 m.

Full size image
Fig. 3.
figure 3

Channel arrangement in fNIRS measurement

Full size image

3.2 Measurement of Eye Movement

We performed our experiment on six healthy males with ages ranging from 23–24 (mean ± standard deviation: 23.3 ± 0.5). Images used for the experiment were recreated based on Sky Crystal (Olympus Memory Works Corp, Tokyo) with permission. For measurements, we used an eye mark recorder, EMR-9 (Nac Image Technology, Tokyo). Images were displayed with Display 55UF8500 (LG, Seoul). Measurements were taken in a darkroom, and subjects were seated for the experiment. Following the 60-s test with eyes closed (Pre), subjects tracked 2D images for 60 s and used peripheral vision for another 60 s. Two types of 2D images (with and without backgrounds) were used.

Tracking and peripheral vision eye movements were recorded with the sampling frequency of 60 Hz and resampled at 20 Hz. The position of the viewpoint for each sampling time during viewing—x-y coordinate [pix]—was measured and each index was recorded. Data were divided into x-direction (the right to the LCD screen is positive) and y-direction (vertically upward is positive), and are converted to a time series. Total locus length, outer peripheral area, and unit area locus length were evaluated. Also, we performed Wilcoxon signed rank test for each analytical index. The significance level is 0.05.

4 Results

We herein report results of the brain activity and eye movements in this study.

4.1 fNIRS Measurements

Figures 4 and 5 show channels, which were compared with Pre through Wilcoxon signed rank test and showed significant difference/trends, in color.

Fig. 4.
figure 4

Chs for which statistical increase in oxygenated hemoglobin was observed while viewing images without background: (a) tracking, (b) using peripheral vision (: p < 0.01, : p < 0.05, : p < 0.10). (Color figure online)

Full size image

When viewing images without background, compared to Pre, oxygenated hemoglobin concentration in the occipital lobe increased regardless of the method of viewing (p < 0.05). The change was more notable when using peripheral vision compared to tracking. Similarly for viewing images with the background, oxygenated hemoglobin concentration in the occipital lobe increased. Compared to Pre, a greater increase in the oxygenated hemoglobin concentration was observed in the upper occipital lobe when using peripheral vision than when tracking (Fig. 5). Increase in oxygenated hemoglobin concentration in the frontal lobe compared to Pre was only confirmed when using peripheral vision for images without background.

Fig. 5.
figure 5

Chs for which statistical increase in oxygenated hemoglobin as observed while viewing images with background: (a) tracking, (b) using peripheral vision (: p < 0.01, : p < 0.05, : p < 0.10). (Color figure online)

Full size image

4.2 Eye Movement Measurements

Figure 6 shows temporal changes in eye movement when tracking. For the peak interval of x-axial components (periodic structure), the difference between subjects is small and there was in-phase variation. Figure 7 shows a typical example of a scatter plot for viewpoint position obtained from measurements. Regardless of the background for the image, when tracking and using peripheral vision are compared, there was a clear difference in eye movement. On the other hand, when the scatter plot for viewpoint position for peripheral vision is compared, there was no difference with the background (Fig. 7b and d). However, when the scatter plot of the viewpoint position for tracking is compared, the viewpoint is focused at the center when there is no background (Fig. 7a and c).

Fig. 6.
figure 6

A typical example of eye movements while tracking (a) without background, (b) with the backgrounds.

Full size image
Fig. 7.
figure 7

Typical example of a scatter plot for viewpoint position: (a) without background/tracking, (b) without background/using peripheral vision, (c) with background/tracking, and (d) with background/using peripheral vision.

Full size image

We compared each analytical index calculated from scatter plots for viewpoints between viewing methods (Fig. 8). For the total locus length, there was no significant difference (Fig. 8a). For the outer peripheral area (Fig. 8b), regardless of the background, values while tracking were statistically higher compared to those measuring the use of peripheral vision. For the unit area locus length (Fig. 8c), values measuring the use of peripheral vision for images without background were statistically higher than those for tracking. Also, values measuring the use of peripheral vision for images with background were significantly higher than those for tracking.

Fig. 8.
figure 8

Comparison of the sway values: total locus length (a), area of sway (b), and total locus length per unit area (c).

Full size image

5 Discussions

In the last conference, we focused on a 3D game machine to increase knowledge about the influence of stereoscopic images on the body. Changes in brain blood flow while playing the game were compared between playing using the 2D and 3D displays by measuring NIRS [10]. Since the visual area responsible for vision is present in the occipital lobe [11,12,13], it may have been strongly influenced when 2D was switched to 3D.

A marked increase in brain blood flow was noted in the frontal, occipital, and left temporal lobes. In the frontal lobe, brain blood flow consistently increased from early 2D to late 2D at many channels. Since the frontal lobe controls psychogenesis, such as emotion, attention, thinking, and voluntary movement, brain blood flow may have continued to increase due to thinking and concentration on the game operation. In the occipital and left temporal lobes, brain blood flow increased from early 2D to 3D, and no significant difference was noted between 3D and late 2D. Various sensory areas are present in the left temporal lobe, and, in contrast to the right temporal lobe which memorizes sounds and shapes, the left temporal lobe memorizes and understands speech [10,11,12,13]. Since TETRiS® is a game requiring thinking, 3D images may have a large influence on the left temporal lobe.

The visual area responsible for vision is present in the occipital lobe, and various sensory areas are present in the left temporal lobe. Thus, sensory areas, such as the visual area, may have been activated when playing the game with 3D images compared to activation by 2D.

5.1 fNIRS Measurements

When there was no background, compared to Pre, values of oxygenated hemoglobin concentration showed more fluctuations when using peripheral vision compared to tracking. However, with the background, there was no significant fluctuation. For images with the background, information from the screen was increased, and subjects became distracted by images other than the target. Regardless of background, when compared with Pre, oxygenated hemoglobin concentration in the upper occipital lobe increased more for peripheral vision than for tracking. If we assume Neuro-Vascular-Coupling (NVC), when using peripheral vision, brain activities in the upper occipital lobe were accelerated. This is because brain activity on the dorsal visual pathway was promoted, which may be useful in examining the mechanism of 3D sickness [14]. Also, increase in oxygenated hemoglobin concentration in the frontal lobe compared to Pre was only observed when using peripheral vision for images without background. The frontal lobe is used for cognition. Since there is no information in this image other than the movement of the target, when using peripheral vision, it especially requires cognition that is associated with spatial recognition to grasp perspective. On the other hand, when tracking movements of the target, information on perspective was input to the frontal lobe through the ventral visual pathway. As discussed above, when using peripheral vision, brain activity in the dorsal visual pathway that controls spatial recognition is promoted, but with the lack of information on the background perspective, subjects supplemented spatial recognition in the frontal lobe through cognition, which increased the oxygenated hemoglobin concentration in the frontal lobe.

5.2 Eye Movement Measurements

In the x-axis components on eye movement, in-phase temporal fluctuations were observed in all subjects (Fig. 6). As such, homogeneous tracking of the visual target could be confirmed.

From the scatter plot of viewpoint location, we were able to confirm viewing methods. When backgrounds were compared for peripheral vision, there was no difference in the scatter plot related to viewpoint position (Fig. 7b and d). However, when backgrounds were compared for tracking, viewpoint position was focused at the center when there was no background in the image (Fig. 7a and c), and eye movement was stable. When using peripheral vision, movement of the visual target at the center of the field of vision did not have an impact.

In the outer peripheral area (Fig. 8b), regardless of background, compared to when using peripheral vision, values were statistically higher for tracking. For unit area locus length (Fig. 8c), when using peripheral vision for images without background, values were statistically higher compared to when tracking, confirming the findings of stable eye movement. Even when information on the background perspective is missing, conditions of peripheral vision do not change; and thus, there is insufficient data for spatial recognition. However, since these data have large scatter, we need to increase the number of subjects for further examination.