Keywords

1 Introduction

A multitude of virtual methods, amongst which driving simulators, are increasingly used in the automotive product development process. An important driver for this is the demand for shorter and more cost-efficient development processes [1, 2]. Driving simulators offer the opportunity to carry out driving tests in a safe, reproducible and controlled environment [3]. The testing scope for driving simulators includes human-machine interactions, such as driver distraction studies for operating concepts, assessment of assistance functions and automated driving functions [4] while also involving aspects like suspension set-ups utilizing hardware-in-the-loop simulations [5, 6].

Despite the continuous improvement, driving in a simulator has been and always will be merely a simulated reproduction of reality. Due to an increasing number of optimizations and a strong dependency between individual components of the driving simulation, a holistic evaluation of further developments in the driving simulation is necessary. With a larger number of applications and an ever advancing technical development, the need to prove the validity of driving simulator tests has been increasing. This requires methods to analyse the differences between simulators and to quantify the effects of improvements in simulator configuration. With the method reported in this paper the influence of the reproduced vertical dynamics on behavioural validity will be examined. The National Highway Traffic Safety Administration (NHTSA) recommends testing on driving simulators especially for driver distraction related studies [7]. Resulting from the need to evaluate, for example, new touch screen control elements, a suitable method for presenting vertical dynamics is necessary. For the implementation of vehicle vibrations, the simulation software was upgraded and a static simulator was equipped with four linear actuators (Fig. 2). The four actuators allowing a simulator motion with three degrees of freedom (3-DOF: heave, pitch and roll).

2 Driving Simulator Validity

Driving simulator validity is an active research topic [1, 8, 9] and an important requirement for transferring development tasks from the road into driving simulator laboratories [10]. Depending on the desired focus of the research, different types of simulator validity are used. According to Blana et al. [11, 12] the term of driving simulator validity can be divided into:

  • Behavioural validity describes the correspondence between driving behaviour in the simulator and on-road.

  • Physical validity measures the extent to which a driving simulator can reproduce the visual and vestibular cues while driving an actual vehicle.

  • Absolute validity is defined as a quantitative criterion which is achieved if the numerical values of these performance differences are about equal or the same.

  • Relative validity is achieved if the difference is in the same direction and relative size of the effect of the measure as in reality.

To examine the validity of the simulator in this study the definition of behavioural validity is used. In this case the behavioural concept refers to the driving performance and the self-reported mental workload.

3 Background and Research Question

The research in [13] provided some evidence that adding a 3-DOF motion system to a static simulator can improve the lane keeping performance and decrease the self-chosen vehicle speed on country roads. Another study showed that adding vibrations to a 3-DOF motion system can alleviate the symptoms of motion sickness [14]. Neither study, however, investigated the influence on behavioural validity. Another approach focused on improving the realism of a driving simulator by presenting vehicle vibrations to the participants [15] and validating the influences on the self-chosen speed [16], but the findings were based on a advanced driving simulator with 6-DOF.

The current paper aims to add to existing knowledge by providing a detailed analysis of the influence of vertical dynamics on behavioural validity on highway conditions in a cost-efficient mid-size 3-DOF simulator.

Reproducing vertical dynamics leads to vehicle vibrations, which are perceived by the driver. On the one hand the presence of vehicle vibrations can be interpreted as an additional cue for the driver and results in an increased immersion. On the other hand the resulting vehicle movements can be interpreted as a disturbance input for the driver-vehicle-environment control loop. To determine the influence of the vehicle vibrations on the driving performance the following research question will be answered:

How does the reproduction of vehicle vibrations on a driving simulator influence the stabilization performance, the-self reported mental driver workload and the self-reported simulation sickness?

4 Methodology

To examine the research questions two empirical studies were performed. In May 2018 a real driving study on an highway near Munich was conducted with 30 participants. This study will be referred to here as the On-Road Study. Furthermore, a driving simulator study in one of the BMW Group’s driving simulation facilities with 41 participants was conducted in August 2018.

4.1 Apparatus

On-Road Study. Figure 1 shows the experimental vehicle used for the on-road study. The used car was a 4-door sedan BMW 5-Series with 248 horse power. For data acquisition the internal vehicle bus was recorded with 20 Hz using a data logging system. At the bottom left on Fig. 1 the experimenter interface is shown. The interface was displayed on a \(7^{\prime \prime }\) display which can be positioned near the passenger or rear seat. The right-hand side of Fig. 1 shows the driver position. The experimental car was equipped with a car radio to perform the selected driving tasks (see Sect. 4.4). The modifications of the experimental vehicle were implemented such that the general appearance for the participants, apart from the radio, is not different from a series-production vehicle.

Fig. 1.
figure 1

Experimental vehicle used for the on-road validation study

Full size image

Simulator. The driving simulator used in this validation study is shown on Fig. 2. The vehicle mockup consisted of the front section of a BMW 5-Series chassis and 5-Series cockpit elements, which was mounted on a electromotive 3-DOF platform. The platform was actuated using 4 D-Box actuators (D-BOX TECHNOLOGIES INC., Gen II Actuators \(6^{\prime \prime }\)). The vehicle dynamic model described the same vehicle type which was used for the on-road study. All typical control elements such as direction-indicator control, brake and acceleration pedal and gear selector lever were fully functional. The steering torque feedbacks came from an electric actuator (Sensodrive GmbH, SENSO-Wheel SD-LC) that delivers a nominal torque of \(7.5\,\mathrm {Nm}\). For the visualisation of the virtual environment five video projectors were used which enable a \(220 \mathrm {^\circ }\) horizontal and \(45 \mathrm {^\circ }\) vertical field of view. The views seen in the exterior and interior rear view mirrors were provided by three LCD screens.

Fig. 2.
figure 2

3-DOF driving simulator system used in the validation study

Full size image

4.2 Vehicle Vibrations

The main aspect of this study was to examine the influence of vehicle vibrations on driving simulator validity.Vehicle vibrations can be classified according to Heissing et al. [17] in three categories. Vibrations with frequencies from 0–25 Hz are perceived by tactile sense and are categorized as tangible. Frequencies higher than 100 Hz are mainly noticed as audible noise and hence categorized as audible. Between these frequency bands exists a so called transition zone. Phenomena in this transition zone can be perceived by both, the acoustic and tactile senses and are therefore both audible and tangible. To present these effects to the participants three types of actuators were used:

  • 3-DOF motion-system (D-BOX TECHNOLOGIES INC., Gen II Actuators \(6^{\prime \prime }\)) for frequencies up to 30 Hz, i.e. the tangible range

  • Structure-borne sound converter (SONIC IMMERSION TECHNOLOGIES, IBEAM VT-200 Transducer) for frequencies from 30–125 Hz, i.e. reproducing roughly the transition zone

  • Four channel sound system mainly for frequencies over 100 Hz up to 20 kHz

The structure-borne sound converter was located under the driver seat to represent phenomena like road roughness, road bumps and engine vibrations. Vehicle vibrations with frequencies up to 100 Hz are mainly caused by tyre-road interactions [18]. For an adequate representation of these effects a realistic simulation of road surfaces is necessary.

Therefore the road surface for the simulated road was derived from laser-measured data of the highway on which on-road study was performed. The measured road excitations are provided to the driving simulation by the OpenCRG file format [19] with a resolution of 10 \(\times \) 10 mm. To obtain the resulting vehicle accelerations and angular rates, a non-linear double-track model is used for the vehicle dynamics simulation. To present the vehicle vibrations to the participants the calculated movements are transferred to the motion-system and the structure-borne converter.

It should be noted that this implementation also allows for the reproduction of steering torque effects caused by road excitations from road damages, such as grooves, potholes and cracks. Furthermore, it has to be taken into account that in the real world road damages are not only perceived as vestibular or tactile cues but also by visually cues. Therefore the existing road damages were also reproduced graphically (see Fig. 3).

Fig. 3.
figure 3

Virtual highway with road damages

Full size image

4.3 Participants

In the on-road study 30 BMW employees (28 men and 2 women), with at least six years of driving experience and a minimal annual mileage of \(5000 \; \mathrm {km}\), took part. The age of the participants ranged from 25 to 50 (mean \((\mu ) = 36.13\) years, standard deviation \((\sigma ) = 8.28\)).

The simulator study was conducted with 41 BMW employees (34 men and 7 women), with at least four years of driving experience and an minimal annual mileage of \(5000 \; \mathrm {km}\). The age of the participants ranged from 20 to 59 \(\mathrm {(\mu = 37.12 \ years, \sigma = 12.08)}\). 31 of 41 participants had prior experience in a driving simulator. The remaining 10 participants took part for the first time in a driving simulator study.

All participants were included on a voluntary basis. The participants received written instructions and provided their informed consent before the experiment commenced. They were informed about that they could abort the experiment at any time in case of discomfort.

4.4 Study Design

Independent Variables. The driving simulation study was designed as a within-subject design with the two independent variables ‘Flat’ and ‘Ride’ combined with the real driving study ‘Real’.

  • Real: On-road study which serves as benchmark for a realistic stabilization performance.

  • Flat: Only roll- and pitch angles induced by vehicle accelerations were presented to the participants by the motion system.

  • Ride: This condition extends the ‘Flat’ condition by representing vehicle vibrations due to road-excitations by the motion system and a structure-borne sound converter. See also Sect. 4.2 for more information.

The driving task consisted of a highway ride in which the so-called radio tuning task was carried out according to the specifications of AAM [20]. In the on-road study participants were instructed to follow a lead vehicle at a distance of \(50 \ \mathrm {m}\). The lead vehicle, driven by on of the experimenters, drove in the right lane at a speed of \(92 \ \mathrm {km/h}\). The written instruction for the participants was to focus on lane keeping and following the lead vehicle with a constant distance. This task will be referred to here as the baseline condition. In order to allow an analysis of relative effects (i.e., relative validity) the above described driving task was also conducted while the driver was instructed to simultaneously perform a secondary task. The secondary task was the so-called radio tuning task (radio). In the radio tuning task, the participants were asked to select a given frequency on a radio, which was installed according to the AAM [20]. The radio which was used for the radio tuning task is shown on the right-hand side in Fig. 1.

The drive to the highway took about 20 min. During this time the participants could familiarize themselves with the car. At the beginning of the highway drive the participants practiced the baseline task for about 5–10 min. The radio task was not practiced before.

After the on-road study the radio was installed into the simulator mock-up. The test procedure in the simulator study was the same as in the real study. Before the test cases have been started the participants drove along urban, rural and highway roads for around 10 min to get familiar with the environment. The lead vehicle on the highway was controlled automatically by the simulation software. The highway section where the on-road study was conducted, was reproduced for the simulation including the laser-measured road surface.

To calculate the performance indicators six repetitions of the radio tuning task and the baseline task with a duration of 60 s were performed. The order in which the participants performed the two tasks was randomized.

Fig. 4.
figure 4

Schematic overview of the study design

Full size image

Figure 4 summarizes the experimental design and provides an overview of the independent and dependent variables of the experiment. In the next section the selected dependent variables will be described.

Dependent Variables. The above described scenario was used for the examination of the stabilization performance, whereby the stabilization task can be divided into lateral and longitudinal stabilization [21]. The lateral stabilization performance was measured with the standard deviation of lane position (SDLP). To calculate the SDLP the distance between the right-hand front wheel and the side stripes was chosen. The longitudinal stabilization performance was measured with the standard deviation of head-way (SDHW) [22, 23], in this case the head-way was defined as the distance between the front and rear bumper of the experimental and the lead vehicle.

In addition the self-reported driver mental workload was recorded with the driver activity load index (DALI).

The DALI is a revised version of the NASA-TLX, adapted to the driving task [24] and consist seven workload dimensions:

  • effort of attention

  • visual demand

  • auditory demand

  • temporal demand

  • interference

  • situational stress

  • total score (consisting of the average of the other dimensions)

Each dimension has been valuated by the participants with a numeric rating scale from 0 (low demand) to 5 (high demand) after each of the three repetitions. Afterwards the average of both submitted ratings was calculated.

Simulator sickness is a common issue in the context of driving simulation and a major obstacle to the use of driving simulators for research, training and driver assessment purposes [25, 26]. Therefore the self-reported simulation sickness was quantified by Kennedy’s SSQ Simulator Sickness Questionnaire (SSQ) [27] to analyse the effect of vehicle vibrations on the participant’s sense of well-being. After data acquisition it is possible to provide simulator sickness scores classified in three symptoms referred to as nausea (N), oculomotor disturbance (O), disorientation (D). Additionally a total score (TS) can be derived from the three symptoms sub scales.

5 Results

To test whether the data meet the requirements for a parametric independent analysis of variance (ANOVA), which are that the residuals are normally distributed and have homogeneity of variances [28], the Lilliefors and Levene test were performed [29]. If the conditions are not fulfilled the significance test was performed using the non-parametric Kruskal-Wallis-Test [30]. For the post-hoc tests the p-values were corrected with the least significant difference method [31].

5.1 Lateral Stabilization Performance

Figure 5 shows the results (mean and 95% confidence interval) for the lateral stabilization performance quantified by the SDLP. On the ‘real’ condition the participants produced the lowest mean for the SDLP followed by the ‘ride’ and ‘flat’ condition. The statistical tests reveals significant differences for the SDLP during the baseline and radio task (baseline: \(F(2) = 6.30,\, p < .01\), radio: \(F(2) = 4.83,\, p = .01\)). Regarding to the post-hoc tests the results for the condition ‘ride’ on both tasks do not differ significantly from the on-road study (baseline: \(p>.30\), radio: \(p>.08\)). These results indicates that the addition of vehicle vibrations are required to obtain absolute validity, i.e., no significant difference with the real condition.

Fig. 5.
figure 5

Bar chart: results for standard deviation of lane position (SDLP) based on 71 participants. \(\mathrm {n_{Real} = 30, n_{Flat} = 21, n_{Ride} = 20}\)

Full size image

5.2 Longitudinal Stabilization Performance

Figure 6 illustrates the results for the longitudinal stabilization performance quantified by the SDHW. For the baseline task the means of all three conditions are similar. For the radio task the mean for the ‘real’ condition is the lowest followed by the ‘ride’ and ‘flat’ condition. The ANOVA for the SDHW on baseline condition shows no significant differences (\(H(2) = 1.16,\, p > .55\)). The AMOVA test for the radio task shows significant differences between the three test conditions (\(H(2) = 8.08,\, p = .018\)). The post-hoc contrasts, however, show no significant differences between the simulator and on-road conditions (real-flat: \(p=.058\), real-ride: \(p=.804\)).

Fig. 6.
figure 6

Bar chart: results for standard deviation of head-way (SDHW) based on 71 participants. \(\mathrm {n_{Real} = 30, n_{Flat} = 21, n_{Ride} = 20}\)

Full size image

5.3 Mental Driver Workload

The mental workload quantified by the DALI total score is shown in Fig. 7). A statistical analysis showed a significant effect of task but no significant difference between the conditions real, ride and flat. For closer investigation the six DALI sub scales were also analysed (see Fig. 8) which showed that the auditory demand for the baseline task for the ‘ride’ condition is significantly higher (\(H(2) = 6.31,\, p = .043\)). Furthermore the temporal demand is perceived higher for the ‘ride’ condition during the baseline task (\(H(2) = 6.15,\, p = .046\)). Regarding the radio task, the descriptive analysis shows also higher values for the auditory and temporal demand but the differences are not statistically significant.

Fig. 7.
figure 7

Box plot: results for DALI total score sub-scale based on 71 participants. \(\mathrm {n_{Real} = 30, n_{Flat} = 21, n_{Ride} = 20}\)

Full size image
Fig. 8.
figure 8

Box plot: results for DALI auditory and temporal demand sub-scale based on 71 participants. \(\mathrm {n_{Real} = 30, n_{Flat} = 21, n_{Ride} = 20}\)

Full size image

5.4 Simulator Sickness

To analyse the effect of vertical dynamics on simulator sickness a Wilcoxon rank sum test for two independent samples was performed. Although the ‘ride’ condition provides an additional vertical movement to the participants, regarding the SSQ sub-scales no significant differences were found when comparing the ‘flat’ and ‘ride’ condition (see Fig. 9).

Fig. 9.
figure 9

Box plot: SSQ-Scores sorted by the four SSQ-Subscales N: Nausea, O: Oculomotor, D: Disorientation and TS: Total Score [27]

Full size image

6 Discussion and Outlook

The results of this approach underline the importance of vehicle vibrations on lateral stabilization performance. Absolute validity could be achieved after presenting vehicle vibrations for the SDLP in both conditions. These results show that the presented additional cues provide useful information for the lateral stabilization task. For the head-way performance (SDHW) absolute validity is reached for all simulator conditions. This means that presenting road excitations to the subject has no negative effect on longitudinal stabilization performance. It will be assumed that the perceived visual cues are sufficient to perform the longitudinal stabilization task.

Although the DALI total score shows no significant differences between the two test conditions - flat and ride - the descriptive analysis indicates an increasing overall mental workload for the ride condition. This increase seems to be caused by a higher temporal and audible demand.

Possible explanations for the higher audible demand could be that the cues, provided by the vehicle vibration system in the ride condition, did not present the participants a realistic experience or caused additional noise that did not aid the drivers in their task. Another explanation may be that the study was conducted with experienced driving simulator drivers which were not familiar with the newly presented cues in this experiment. The possible correlation between the increased audible demand and temporal demand is to be investigated.

A 3-DOF motion system in combination with vehicle vibrations delivers a cost-efficient solution to improve the lateral stabilization performance without influencing the longitudinal stabilization performance. An negative effect regarding to the SSQ could not be found. A possible next step is to investigate the specific reasons for the higher demanded mental workload while driving with simulated vehicle vibrations.

To analyse the effect of vehicle vibrations on motion simulators with more than 3-DOF the study should be repeated in classical motion simulators with a hexapod motion base. Furthermore this approach will be extended by a country road scenario to evaluate the implementation of vehicle vibrations, with more behavioural aspects. In this future study additional behavioural aspects, such as a subjective evaluation of the realism of the driving scenario and perceived driving speed, will be included.