Keywords

1 Introduction

Virtual reality (VR) is a computer-based environment that simulates realistic experiences [1]. As the progress for the development of VR, the market for head-mounted displays (HMD) like HTC Vive [2], PlayStation VR [3] and Oculus Rift [4] has grown rapidly. HMD is a wearable device attached to the head and supports users’ head movement tracking. This makes it easier for the user to look around virtual scenes along with the head movement. Although HMD offers high reality in visual circumstances, other modalities such as haptics play a key role for apprehending the environment [5]. Haptic display devices such as SPIDAR [6] and Geomagic Phantom [7] have been developed to present high-reality haptics in VR environment.

According to Tachi et al. [1], there are three essential elements to establish the appropriate VR environment.

  • Autonomy: building a natural three-dimensional space

  • Interaction: immediate and prompt reflection for the users’ behavior

  • Presence: feeling like you are present inside the VR environment

The autonomy and the interaction can be determined by the performance of computers to construct the VR. In contrast, a method for quantifying the presence has not been established because it varies depending on human senses and physiological responses. In VR environment where the presence is limited, users may feel stress, yielding frequent operational errors. For instance, in the telerobotic surgery, if an operator does not perceive force feedback correctly through the haptic display, this can cause fatal medical accidents [8]. Therefore, it is important to quantify the presence for preventing users’ excessive stress and the potential accident due to lack of accurate presence.

The presence has attracted a lot of attention and research regarding the presence has flourished in recent years [9, 10]. In addition to the subjective questionnaire, evaluation for the presence using physiological indices such as electromyogram (EMG) and heart rate have attracted attention [11, 12]. We focused on EMG as a physiological index to evaluate the presence. In the previous study, EMGs when catching a free-falling ball in the haptic VR environment have been evaluated [13]. They focused on the start time of activation of the muscles in the forearm before catching a ball. However, they did not compare EMGs in VR environment and that in real environment. If the presence decreases with inappropriate haptic presentation, different muscular activities from the real exercise may be generated. Therefore, it is speculated that the temporal characteristics on muscular activities were the potential index for evaluating the presence quantitatively.

Therefore, the objective of this study was to construct the system for demonstrating haptic VR environment and to conduct comparison of EMGs during catching a ball in VR and real environments.

2 Methods

2.1 Participants

A total of 3 males participated in this study. Two were in early their twenties and one was in his early fifties. They were right-handed. They have experienced VR several times, but not regularly.

2.2 Experimental Setup

Figure 1 shows the overview of the experimental set up. The experimental setup was composed of VR environment presentation equipment and EMG measuring instrument. HTC Vive and space interface device for artificial reality – human scale (SPIDAR-HS) were used for constructing VR environment.

Fig. 1.
figure 1

Overview of experimental setup.

Full size image

HTC Vive is shown in Fig. 2. The frame rate of HTC Vive was 90 frames per second. HTC Vive has a field of view of 110 degrees, slightly less than typical viewing angle of the human eye (approximately 120 degrees). The resolution of HTC Vive is 1080 × 1200 per eye [2]. HTC Vive wearer can see the VR space created by Unity.

Fig. 2.
figure 2

HTC Vive for visual presentation in VR environment.

Full size image

SPIDAR-HS is derived from the original SPIDAR which was introduced in 1991 by Sato and colleagues [6]. After that, Scaleable-SPIDAR [14] was developed as a predecessor of SPIDAR-HS in early 2000. SPIDAR-HS is 2.5 m × 2.5 m × 2.5 m in size. Within this space, different kinds of force feedback senses associated with weight, contact and inertia can be displayed to the user’s hand by tensioned wire. The device used tensioned wire techniques to track hand position as well as to provide haptic feedback sensations. The force feedback felt on the user’s hand was the same as the resultant force of tension generated by the collection of eight wires equipped at each corner of the frame. In order to control the tension and length of each wire, one end was connected to an end-effector and the other end was wound around a pulley, which was driven by a DC motor. By controlling the power applied to the motor, the system was able to create designated tensions all the time. A rotary encoder was attached to the DC motor to detect the wire’s length variation. The updating frequency of the motor was set to 500 Hz. Maximum force was 6.30 N. A half-sphere-shaped end-effector, shown in Fig. 3, was made with a 3D printer. The set of DC motor, pulley and encoder controlling each wire was fixed on the frame. The frame was made of aluminum (HF S6-3030-2500). Figure 3 shows the structure for SPIDAR-HS.

Fig. 3.
figure 3

SPIDAR-HS for haptic presentation in VR environment

Full size image

Muscular activity was measured with a bioamplifier (BA1104 m EMG, Nihon Santeku, high-pass frequency of 1000 Hz, TC: 0.03 s). EMG signals were recorded with a sampling rate of 1000 Hz and stored at the data logger (GL900, GRAPHTEC). In this study, we used the surface electrodes (BA-U410 m, Nihon Santeku) for measurement of EMG signals. Also, we used an electrode (M-00-Sm, Ambu) and a cord (YCE116 m, Nihon Santeku) as a body ground for the measurement of EMG.

2.3 Experimental Protocol

Throughout the experiments, the participant was sitting on the chair with the right forearm placed on the armrest with the right palm facing upwards. The muscular loads for two muscles in the right forearm areas were registered by electromyography. Participant was applied electrodes to the palmaris longus muscle (PLM) and the extensor carpi radialis longus muscle (ECRLM) in the right forearm (Fig. 4). PLM manipulates the palmar flexion of the wrist and ECRLM manipulates the wrist to move the hand away from the palmar side [15]. The electrode for the body ground was attached to the right elbow. EMGs were measured while participant was at rest, during ball catching task in real environment (Real task) and during ball catching task in VR environment (VR task).

Fig. 4.
figure 4

Surface EMG electrode positions. Palmaris longus muscle (PLM) and extensor carpi radialis longus muscle (ECRLM) are in forearm

Full size image

Measurement of EMG at Rest.

Participant’s EMGs were measured for 15 s at rest. We instructed participant not to exert on his right arm. This task was executed once before the Real task. If it was confirmed that the muscles were activated during this measurement, the measurement was retried.

Real Task.

In Real task, participants were asked to perform catching free-falling ball sequence in real environment. The initial position of the ball was 80 cm above the right palm. EMG measurement was started when the ball began to fall. When catching a ball, participant was instructed not to grasp the ball with fingers but to catch as much to support it with the palm as possible. This task was carried out three times. Figure 5 shows the ball used at the Real task. The ball was 220 g and its diameter was 80 mm.

Fig. 5.
figure 5

Ball used at Real task. The weight was 220 g and diameter was 80 mm.

Full size image

VR Task.

Participant attached HTC Vive to the head. The initial position of the virtual ball on the HMD’s display was 80 cm above the right palm. EMG measurement was started when the virtual ball began to fall. SPIDAR-HS presented haptic information to the hand at the moment the virtual ball reached to the palm (see Fig. 6). When catching the virtual ball, participant was instructed not to grasp the object with fingers but to catch as much to support it with the palm as possible. This task was carried out three times.

Fig. 6.
figure 6

Integration of vision and haptics in VR environment. Free-falling ball was projected on HTC Vive display. Haptics as ball impact force was presented by SPIDAR-HS.

Full size image

2.4 Data Analysis

LabVIEW2017 (National Instruments) was used for data analysis. For the analysis of EMG data, band-stop filtering 49 Hz – 51 Hz and band-pass filtering 10 Hz – 350 Hz were used, followed by root mean square (RMS) processing.

The moment when the ball began to fall was taken as zero second. Catching Onset Time is the start time of activation of muscles in the forearm before catching a ball. Catching Onset Time was determined by rectified signals exceeded the maximum value at rest. Time when the ball contacted the palm (Contact Time) was calculated from Eq. (1) that was based on Newton’s law of motion. Let the time when the ball contacted the palm be \( \varvec{ }t \), gravitational acceleration be \( g \) and the height of the ball from the palm be \( h \).

$$ \varvec{t} = \sqrt {\frac{{2\varvec{h}}}{\varvec{g}}} $$
(1)

3 Results and Discussion

3.1 Results of Experiments

Figure 7 shows the EMGs during the ball catching task in VR and real environments for three participants. In the figure, the up arrow shows Catching Onset Time and the down arrow shows Contact Time. It was not successful to measure ECRLM activity of participant C at VR task due to excessive motor noises contaminated to EMG signals.

Fig. 7.
figure 7

EMGs during ball caching at VR and Real task for three participants. The vertical line shows the muscular activity and the horizontal one shows elapsed time from the ball began to fall. Left: PLM, Right; ECRLM.

Full size image

3.2 Tendency of Muscular Activity

Figure 8 shows correlation scatter diagrams of the EMG for each Real VR tasks from Catching Onset Time to Contact Time. Correlation coefficient is also shown in each diagram. The high correlation coefficient means that the tendency of the muscular activity at VR task and at Real task is similar.

Fig. 8.
figure 8

Correlation of the EMG signals between real task and VR task from catching onset time to contact time. Left: PLM, Right; ECRLM. The high correlation coefficient (r) means that the tendency of the muscular activity at VR task and at real task is similar.

Full size image

High correlation coefficients (r > 0.9) were found in PLM of participant B and C, and in ECRLM of participant A and B. Relatively high correlation indicated that the tendency of the muscular activity at VR task and at Real task was similar in most cases. ECRLM of participant C showed low correlation coefficient (r = 0.334) due to incomplete measurement at VR task. PLM of participant A showed slightly lower correlation coefficient (r = 0.770) than the other conditions. It was interpreted that when Catching Onset Time was determined, time delay was occurred between VR and Real tasks. The method of establishing Catching Onset Time in this experiment was the moment of exceeded the maximum rectified EMG signals at rest. According to Ishida et al. [16], there were three methods to establish Catching Onset Time and further analysis should be applied with specific onset time estimation methods.

3.3 Amount of Muscular Activity

Average peak RMS value for forearm EMG at VR task was 45.2% smaller than that at Real task. There were two probable causes for this difference. First, the amount of force generated by SPIDAR-HS at VR task was relatively lower than that made by the gravity force of the ball at Real task. Therefore, less muscular strength of the forearm was required for VR task compared to Real task. Second, participants seemed not to flex the forearm very well at VR task because they did not require as much effort to catch the ball as the Real task. For instance, even if they caught the ball with edge of the palm, the virtual ball stayed securely at the palm regardless of the precision of hand location. On the other hand, unless they performed the catching task precise enough such that their palm stayed exactly beneath the center of mass of the falling ball, the ball dropped to the floor at Real task. Therefore, muscle of the forearm was not strong enough at VR task compared to Real task.

As the amount of muscular activity changed depending on the VR environment, it was indicated that it can be used as an evaluation index of the presence. It was necessary to adjust the amount of force generated by SPIDAR-HS and to correct the ball-falling program for further evaluation of the presence using VR task.

4 Conclusion

In this study, we constructed the haptic VR environment whose visual presentation by HTC Vive, and haptic presentation by SPIDAR-HS. Also, we compared the dynamic muscular activities of the forearm during the ball catching task in VR and real environments, and investigated whether dynamic muscular activity would be an evaluation index of the presence of VR. Dynamic changes in the amount of muscular activity confirmed that the average peak RMS value for forearm EMG at VR task was 45.2% smaller than that at Real task. The difference in force magnitude was present between the tasks in VR and real environments. SPIDAR-HS could only generate a force less than the gravity of the ball. The tendency of muscular activity confirmed that the muscular loads from the preliminary motion for ball catching to the completion of the catch was similar between VR and real environments. Therefore, it was concluded that the presence of VR environment was observable by changes in dynamic muscular activities during VR tasks.