Keywords

1 Introduction

Augmented Reality (AR) display development in the automotive industry is proceeding and will probably be approaching series development in the next decade. So far, some use cases as distance indicators for parking are already implemented in the center display [1]. However, future concepts will address windshields as display location, where even more application scenarios are possible. Besides many advantages (e.g. higher usability of information, less visual distraction) there are also disadvantages. Kim [2] especially mentions driver attention issues. Because of their high salience [3], high rate of information switches [4] or visual clutter [5] interaction with these displays could result in decreased driver attention. Further problems concern the coverage of real world objects that also could result in driver inattention issues.

To avoid these possible disadvantages and fulfill future user requirements, designers and researchers are working on various solutions. This paper describes exemplary design factors for the application scenario “navigation with AR in the vehicle”. First, we present a selection of related work in this sector. Second, our conducted experiment provides data about the effect of coverage by AR visualizations as well as about the comparison of different visualizations.

The contributions of our work are as follows:

  1. (a)

    Traffic safety: Insights about the effect of real world object coverage by AR visualizations on driver performance.

  2. (b)

    Design: Insights about acceptance of different design alternatives for AR navigation in the vehicle.

2 Related Work

2.1 AR in Vehicles

AR is a field of technology that is spreading over a huge variety of different application areas. For a good overview of areas and research conducted, please see Dey et al. [6] and their systematic review of AR usability studies. In our work, we focus on the application area “navigation and driving” and use the term AR visualization analogue to Bubb’s definition of contact analog displays [7, p. 281]: “a real representation of the environment, where artificial information is imbedded”. This information is displayed at the same time and the exact location of real objects. First applications are dated from the mid-seventies [8] and were implemented in a real driving vehicle by Assmann [9].

The main goal of these applications is to reduce cognitive effort when interpreting information that is displayed by the in-vehicle information system and to increase situation and system awareness [10].

Until now, a lot more possible use cases have been proposed. For instance, Haeuslschmid et al. [11] present 96 different use cases that are suitable for windshield display applications and often involve some sort of augmentation. When introducing their design space, they present four different categories of application purposes: safety, vehicle monitoring, entertainment and navigation & geo information systems.

An example for safety applications is the marking of pedestrians and objects [12], whereas supporting conditionally automated driving [13] is an example for the category vehicle and monitoring. Social interaction with other drivers would belong to the entertainment category. However, a huge amount of application examples in the vehicle belongs to the category of navigation and geo information systems. Examples are intersection assistants [14] or navigation visualizations [15].

2.2 AR Navigation in Vehicles

The navigation task itself consists of wayfinding and motion as well as the linked cognitive and motor elements [16]. For more information about stages and processes, please see Düner et al. [16].

Narzt et al. [17] list some of the advantages, when displaying navigation information with AR in the vehicle: conventional navigation systems show only abstractions of navigation data (e.g. flat arrows indicating a turn), so the information presented is not clear and users have to abstract this information. AR eases this by showing clear information with no need for abstraction at the relevant location. Furthermore, it helps at situations with a higher level of ambiguity, for instance turning left when there are several junctions in a row and users have to count junctions with a conventional system. In addition, AR also could show information that is obstructed by other vehicles in the driver’s view.

In the past years, researchers and designers presented various designs to show navigation information with AR. Those designs mostly could be divided into following categories:

  1. a.

    arrow (e.g. Pfannmüller [10]): route painted in form of an arrow

  2. b.

    tube (e.g. Narzt [17]): route painted in form of a tube or carpet

  3. c.

    boomerang (e.g. Pfannmüller [10]): route painted in form of multiple boomerangs

  4. d.

    virtual cableTM (e.g. [18]): route painted in form of a line above the street-level activity of traffic

  5. e.

    landmark indications (e.g. Bolton et al. [19]): route events (e.g. turns) marked by an arrow or a box that is highlighting a landmark

  6. f.

    virtual follow-me car (e.g. Topliss et al. [20]): route presented by a virtual lead-car

When investigating these designs, we find several factors that are varied between those visualizations. In the following, we list some of those factors and introduce relevant research.

  • Metaphor. Besides a traditional driving trajectory, there are other metaphoric approaches to guide drivers through the streets. For instance, Bolton et al. [19] evaluated trajectory arrows against landmark arrows and landmark boxes. The concept of landmark navigation resulted in improved decision times and success rates compared to arrow visualizations. They suggest that this is due to less ambiguity when navigating with landmarks. Topliss et al. [20] introduce another kind of navigation metaphor: the virtual lead vehicle. In their studies, it performed especially well when navigating at complex situations such as complex junctions.

  • Display concept. Especially visualizations that are following the trajectory metaphor are varying regarding their specific design (e.g. arrows, tube or boomerang). Pfannmüller [15] tested arrows against boomerang navigation. Here, the arrow was rated worse in terms of clearness and was rated as more stressful than the boomerang visualization.

  • Color. Several factors influence perception of AR colors, for instance different backgrounds and lighting conditions. Gabbard [1] describes these problems in detail. Solutions are for instance adaptive AR concepts, that consider those problems and adjust the AR graphics in real-time. In general, Merenda et al. [21] found out that blue, green, and yellow AR colors are robust to different background colors. Considering user preferences, Pfannmüller [15] found out, that users wish a simple, discreet design with less salient colors to avoid too much attention capturing by the visualization.

  • Presence. Another important detail is the duration a visualization is visible (e.g. permanent or maneuver-based). Milicic [22] explains that a permanent presentation of these visualizations could lead to driver distraction in terms of cognitive capturing, a form of unconscious shift of attention from the road to the AR information, and therefore should be considered carefully.

  • Shapes. Another design question is how to design the edges of a visualization. For instance, Tönnis et al. [23] evaluated three different arrow types: solid arrows with hard corners, flat arrows and solid arrows with rounded shapes in terms of their perception for larger distances. They found out, that especially rounded shapes reduce perception in large distances.

  • Tilt. To achieve a higher salience the visualization could also be tilted along the longitudinal axis when driving towards a maneuver point. When evaluating this design against other design variants, Pfannmüller [10] found out, that a tilted version is ranked worst. Presumably, this is due to a higher perceived distraction because of its high salience or user problems when interpreting this concrete design ahead of maneuvers.

  • Area. Another question is whether to cover a larger area with the visualization to increase salience and clearness of the navigation indication or reduce the covered area to avoid driver distraction issues by misleading attention and covering relevant traffic objects. Horrey et al. [24] speak of a possible degradation of the driver’s ability to respond to truly unexpected traffic events when information covers the driver’s line of sight. Pfannmüller [15] observed that in cases where the lead vehicle was covered by the augmented visualization the contact-analogue impression was destroyed in some cases.

3 Research Questions and Hypotheses

We learn from the previous section, that there is some research concerning the concrete design of AR navigation visualizations. However, some questions remain unanswered, especially when considering specific details of the design. One example is the before mentioned area of coverage of the navigation indication. Researchers stated that coverage per se is not preferable. The simplest solution to this problem would be to reduce the visualization area itself to lower the possibility of any coverage. On the other hand, this approach could lead to a salience that is too low, so the visualization will not be perceived anymore. Overall, our research tackles the question whether the degree of coverage will have any effects on driving performance or subjective perception. Our research hypotheses are as follows:

  • H1: Performance decreases with increasing coverage.

  • H2: Subjective acceptance decreases with increasing coverage.

4 Method

4.1 Study Design

The study follows a within-subject design, exposing all participants to following selected navigation visualizations and selected driving scenarios (independent variables).

In order to manipulate the degree of coverage, three different visualizations (AR tube max, AR tube min, AR two lines) are designed (also see Fig. 1). Concerning the metaphor we choose the most common visualization form of a trajectory path in front of the driver. We choose blue as one of the recommendations from Merenda et al. [21]. Visualizations are not shown permanent but are presented before and shortly after occurring maneuvers in order to avoid any negative effects reported by Milicic et al. [22]. Furthermore we select a flat design for the edges of the AR graphics, to allow good perception even in higher distances (in accordance with Tönnes et al. [23]). Our visualizations are not tilted, because we want to avoid any halo effects due to a large salience because of the tilt effect.

Fig. 1.
figure 1

Overview of evaluated navigation visualizations (from upper left to lower right: AR tube max, AR tube min, AR two lines, AR boomerang, head-up (HUD) conventional)

Full size image

Besides the three visualizations relevant to our research hypotheses, we add two more visualizations to allow a comparison to other common visualizations present in literature (AR boomerang, HUD conventional). The final set of visualizations is presented below:

  • Tube with maximum coverage. This visualization is selected because of its high degree of coverage. It consists of a tube-shaped blue layer that covers most of the upcoming route. This approach allows maximum visual attention for the route guidance with the disadvantage of covering many parts of underlying objects. Similar visualizations can be found in [17] and [25].

  • Tube with minimum coverage. This visualization represents the counterpart to the tube with maximum coverage. It shows the same tube-shaped blue layer but does not cover other moving objects. Hereby the effect of coverage on inattentional blindness should be investigated.

  • Tube with sides. This visualization is a mix of tube with and without coverage. It is reduced to two lines, each one on the left and right edge of the tube. These edges cover other moving objects. So there is coverage, but more reduced than in the version with maximum coverage.

  • Boomerang-shaped. This visualization is similar to the design in Pfannmüller [15]. It consists of multiple arrows in-line, which represent the driving direction. This has the advantage that real objects are less likely to be covered by the visualization because of the gaps between the arrows. When driving towards curves, the arrows also tilt on the longitudinal axis in order to allow higher salience for the upcoming route.

  • Standard HUD. This visualization is added as a comparison to the augmented designs. It represents standard navigation information in form of arrows, which can be seen in front of the driver but are not located at the exact position of the event.

To experience the different visualizations in an interactive way, we choose a driving simulation with different scenarios of navigation. Overall, three scenarios per visualization are driven by each participant: (a) a scenario that consists of a right turn, (b) a scenario with stopping at a red sign and then turning left with oncoming traffic and (c) a scenario with an abruptly braking leading vehicle. Those scenarios are selected because of their high presence in daily traffic, and especially (b) and (c) for the involved traffic and the provoked coverage of other cars by the visualizations.

To avoid sequence effects, we present the visualizations as well as the scenarios in random order. The randomization is carried out with the latin square method.

As dependent variables, we measure various dimensions as performance-based indicators and subjective measurements. For an overview over recorded dependent variables, please see Table 1.

Table 1. Recorded dependent variables assigned to hypotheses
Full size table

To prove, if performance decreases with increasing coverage, brake reaction times are recorded. This measurement is implemented because of Horrey’s et al. [24] research finding concerning decreased performance at unexpected traffic events. For general performance the navigation performance in form of finding the intended way of driving is used.

In order to investigate the effect on subjective perception, the designs are ranked and parts of the User Experience Questionnaire (UEQ) [26] are asked. Due to time constraints, we only use the dimensions perspicuity and attractiveness, as we suppose that differences between the visualizations would mostly express on these two dimensions. After experiencing all visualizations, we ask if participants noticed any form of coverage of objects in the driving simulation. In case a coverage is noticed, the degree of disturbance is ranked for each design.

4.2 Apparatus

The experiment takes place in the driving simulator of Porsche AG with motion dynamics. The simulator has a 6 degrees-of-freedom hexapod system. The platform is equipped with a high-fidelity mockup of a vehicle interior. The visualization system consists of a 4-sided cave with 11 projectors (120 Hz with WQXGA resolution) [27]. An image of the driving simulator can be found in Fig. 2.

Fig. 2.
figure 2

Driving simulator of Porsche AG with motion dynamics

Full size image

As driving simulator software, we use VIRES Virtual Test Drive (VTD) 2.1. The AR navigation visualizations are integrated into the driving simulation as transparent layers on the objects of the driving simulation.

Driving data are recorded and directly transferred to a D-Lab 3.5 software suite. The sampling is carried out at a rate of 60 Hz.

4.3 Tasks and Procedure

After a first instruction of the experiment as well as the review and signing of a participant consent form, a short demographic questionnaire followed. Then participants entered the vehicle mockup and adjusted all relevant controls. Subsequently data recording started and subjects drove a first training phase to get used to the driving simulator. Participants drove about 700 m of country roads and then entered a city, took several turns and then drove another 700 m via country roads to the next city. There the first set of visualizations was presented.

The visualization of the routing started as soon as the upcoming intersection was visible and ended about 50 m after turning into the next road. From first to last appearance of a visualization type, the drive took approximately between 2 and 2.5 min. An exemplary sequence is shown in Fig. 3. Subsequent, participants stopped at the town exit to fill out a short questionnaire. After experiencing all visualizations, participants completed a post-scenario questionnaire and then finished the experiment. Overall, one experiment took about 45 min.

Fig. 3.
figure 3

Exemplary scenario sequence consisting of right turn, left turn with upcoming traffic and suddenly braking vehicle (events randomized)

Full size image

4.4 Participants

Overall, 36 employees of Porsche AG participated in the study, that were recruited via newsletter at Porsche AG in Weissach, Germany. Seven participants dropped-out because of simulator sickness, which could be explained due to the moderate difficulty of the driving task with several turns and braking maneuvers. Another ten experiments were affected by data logging issues in one or more of the trials. These data logging issues were due to problems with our data recording software suite that stopped recording in some cases. This did not influence quality of any remaining cases.

In total, the final dataset consisted of 19 participants. 68.4% were male, 31.6% female. Five participants were between 18 and 24 years, eight between 25 and 39 years, three between 40 and 54 and three participants were older than 54 years. No participant was connected to HUD development and only one subject had a vehicle with HUD system. Three subjects already had driving simulator experience. The study design was approved by a committee consisting of experts from human resources, data privacy and members of the work council.

5 Results

To examine differences between the visualizations, repeated measurements general linear models (GLM) were used (SPSS Version 25). Before analysis, data was checked for any inconsistencies.

5.1 H1 “Performance Decreases with Increasing Coverage”

Figure 4 presents the values of the brake reaction times during the different visualizations in form of boxplots. Means range from 0.92 s (conventional HUD) to 1.09 s (two lines AR HUD). There are no significant differences between the various visualizations (F value: 0.936; sig.: p = 0.449; partial eta sq.: 0.053).

Fig. 4.
figure 4

Boxplots of brake response times

Full size image
Fig. 5.
figure 5

UEQ results (scale ranges from −3 (horribly bad) to +3 (extremely good))

Full size image

Overall performance was observed by counting navigation errors. Statistics are presented in Table 2. With the “tube max” and “two lines” visualization, five participants took the wrong way. With “tube min”, only one participant took the wrong turn. Boomerang and conventional were error-free. In total, there are significant differences concerning navigation errors between the visualizations (F value: 3.816, sig.: p = 0.007; partial eta sq.: 0.145).

Table 2. Overview of navigation errors
Full size table

5.2 H2 “Subjective Acceptance Decreases with Increasing Coverage”

Due to time constraints, only selected items of the UEQ were asked: one item “good-bad” for the dimension attractiveness and all items for the dimension perspicuity. Results are shown in Fig. 5. Although mean ratings differ, those results are not statistical significant, neither on attractiveness (F value: 2.257, sig.: p = 0.069; partial eta sq.: 0.091) nor on perspicuity (F value: 2.219, sig.: p = 0.073; partial eta sq.: 0.090).

Fig. 6.
figure 6

Mean Subjective ranking (1 = ranked best, 5 = ranked worst)

Full size image

Participants carried out a ranking after experiencing all of the visualizations. Results can be seen in Fig. 6. There are significant differences between the rankings of the visualizations (F value: 5.680, sig.: p = 0.000; partial eta sq.: 0.202). The conventional head-up visualization is ranked best with a mean rank of 2.05, followed by the boomerang concept with a mean rank of 2.31. Tube with minimum coverage follows with a mean of 2.63 and tube with maximum coverage with a mean of 2.95. Least ranked visualization is “lines” with a mean rank of 3.63.

Fig. 7.
figure 7

Mean perceived disturbance of coverage (1 = not disturbing, 7 = very disturbing)

Full size image

In the end, participants were asked if they noticed any form of coverage. 32% (6 participants) stated, that they perceived the coverage of simulation objects by navigation visualizations. Those participants rated the degree of disturbance resulting from this coverage for each design. Results are presented in Fig. 7. Perceived disturbance is rated highest for tube with maximum coverage and the boomerang design. Disturbance is rated lower and similar for the remaining designs.

6 Discussion

In the described study, different AR navigation visualizations in the vehicle were compared regarding their degree of covering real world objects as well as their general subjective perception. In order to experience the different designs, participants drove in a driving simulation with each visualization in three different maneuvers.

Concerning hypothesis 1 “Performance decreases with increasing coverage.”, we found ambiguous results. To test the hypothesis we selected two different measurements: brake reaction times to unexpected events as well as the observation of navigation errors. There were no significant differences regarding brake reaction times. However, navigation errors when driving with each visualization differed significantly between the variants. There were more navigation errors when driving with the variants with highest (AR tube max) and moderate coverage (AR two lines) compared to the variant with minimum coverage (AR tube min). This could lead to the assumption that coverage has an effect on the ability to navigate correctly. On the other hand, the comparison variants “AR boomerang” and “HUD conventional” had also some sort of moderate coverage but performed very well when navigating through the streets. So we tend to reject the hypothesis and assume that differences concerning the navigation performance arise from another factor then coverage. The variant “AR two lines” had a very low salience, because of its minimal area of visualization; this could have lead to perception problems in this case. Further explanations might consider usability of the specific designs that contributed to the navigation performance while using.

To evaluate hypothesis 2 “Subjective acceptance decreases with increasing coverage.” we selected three different measurements: the UEQ scales attractiveness and perspicuity, a ranking of the visualizations and a rating that asked for the disturbance that resulted from coverage. Concerning the UEQ scales, we found no significant differences although attractiveness of the variants “HUD conventional” and “AR boomerang” was rated higher in comparison to the other variants. When ranking the variants there were larger and significant differences: “HUD conventional” and “AR boomerang” were rated best followed by “AR Tube min” and “AR Tube max”, last-ranked was the variant “Lines”. In the following questions, participants were asked, if they recognized any sort of coverage. About one third of the participants recognized the coverage of real world objects and rated the perceived disturbance of this coverage. By trend, the disturbance of “AR tube max” and “AR boomerang” was rated higher than that of the remaining variants.

Overall we could state, that rating differs between the variants but we cannot explain this by the varying degree of coverage. Interesting is, that besides all advantages of AR the conventional visualization is ranked best. We explain this by the fact, that users in many cases react negatively to novel devices and do not see the advantages at first glance [e.g. 28]. Another interesting result is that only about one third of the sample recognizes coverage at all.

7 Conclusion

Overlaying real world objects by head-up display visualizations could result in negative effects, which has been reported by Horrey et al. [24] and Pfannmüller [15]. In our experiment, we found differences between the five tested visualizations that were varied by their degree of overlaying real world information. Those differences concerned navigation performance as well as the subjective ranking of the designs. We cannot explain these differences solely by their degree of coverage. However, practitioners could use our insights on subjective preferences of different designs to develop their visualizations in a user-centric way.

Limitations to this work concern the realization of the visualizations. They were integrated into the driving simulation as additional layers, which is not precisely augmented but virtual reality. The setting of a driving simulator was chosen on purpose in order to better realize large visualizations without technical restraints as well as minimizing any interfering factors. Future work should transfer those results in a real driving scenario.

Furthermore, our experiment was realized as a practical setting with several design differences between the visualizations that are interesting but maybe did not solely vary in terms of coverage. In our future work, we want to eliminate any confounding variables and focus on coverage itself. One possible approach would be to vary the degree of transparency of the visualization instead of varying the area of coverage that resulted in quite large conceptual differences.