Keywords

1 Introduction

1.1 Autonomous Driving

While only a few decades ago, self-driving vehicles were more of science fiction than realistic prospect, the topic now has become a determinate and very near future. In fact, partly automated vehicles allowing the driver to temporarily pass the driving task to the automated system and only monitor the traffic and system state are commonplace on today’s roads. Tesla firstly released the Model S equipped with an “Autopilot” in 2015 (Your Autopilot has arrived 2015), quickly followed by other car manufacturers (e.g., The Mercedes-Benz Intelligent Drive system offers compelling tech advances 2015). Automated driving systems have also been released for heavy traffic: In 2015, the partly automated Freightliner Inspiration was introduced in North America (Stromberg 2015).

Driver distraction and, especially in heavy traffic, sleepiness due to monotony of driving state serious risks for traffic safety (e.g., Young and Reagan 2007; Sagberg et al. 2004). The increasing automation of the driving task is supposed to reduce these risks by decreasing the driver’s workload and by overcoming natural human limitations of attention, readiness to react and reaction quality. Simultaneously, the driver’s convenience is addressed with the proceeding of technical development, new opportunities for entertainment, productivity, and recreation. For instance, the safe accomplishment of watching movies, playing games, writing e-mails and reading will be possible in the near future.

The SAE International taxonomy of autonomous driving functions (SAE 2018) describes six levels of motor vehicle automation and has reached mainstream and official recognition.

Level 0 (no driving automation) vehicles refer to ‘conventional’ cars and trucks which are fully operated by the driver.

In level 1 vehicles (driver assistance), the driver is supported by advanced driver assistance systems (ADAS) which temporarily either take over the lateral or the longitudinal subtask.

Level 2 refers to the already mentioned partial driving automation, which requires the driver to permanently monitor the traffic environment as well as the automated driving system which completes both, longitudinal and lateral vehicle control in definite situations.

Vehicles with level 3 automation systems (conditional driving automation) further release the driver from his monitoring task, allowing him to get involved in non-driving-related activities. However, the readiness to resume vehicle control after a sufficient time budget has to be ensured at any time. Note that the opportunity to attend to non-driving-related activities reflects only the system design but not necessarily the legal situation. The current research and developmental focus of automotive manufacturers lies on the imminent market launch of conditionally automated cars and trucks. Besides the technical realization of level 3 automation, the psychological aspects of driver behavior and experience during automation are of major research interest (see Sects. 1.2 and 1.3).

In SAE level 4 (high driving automation), the automated systems fully completes the driving task in a particular range of situations without any expectation that the driver will intervene. The driver is only obliged to intervene in foreseen tasks in specific driving situations which are not included into the particular range of automated situations.

The limitation of application to specific use cases is suspended in level 5. Here, the automated system is required to accomplish the driving task anytime and in any situation. No driver is needed any longer.

Contrary to the SAE taxonomy and nomenclature, many researchers use the term ‘highly automated driving’ when they refer to SAE level 3 (conditional automation). This stems from alternative taxonomies as by the American National Highway Traffic Safety Administration (NHTSA 2013) or the German Federal Highway Research Institute (BASt; Gasser and Westhoff 2012) which have been established earlier than the SAE standard. Also in the current study, ‘highly automated driving’ denotes SAE level 3.

1.2 Behavioral Adaptation

While a few decades ago cars equipped with assistance systems were rare occurrences, advanced driver assistance systems are commonly used these days and drivers rely on them even in situations where a system is at risk to fail (Itho 2012). Manual driving without any assistance systems involves the risk of driver distraction and sleepiness due to monotony of driving state, whereas assistance systems are supposed to reduce these risks by decreasing the driver’s workload and by overcoming potential attentional deficits. They promise relief and relaxation to the driver, they inform them in case of critical traffic situations and release them from control tasks (Sullivan et al. 2016). But with regard to traffic safety, assistance systems can also bear a risk as driver’s familiarity with the systems increases. According to Sullivan et al. (2016), behavioral changes in driver’s behavior take place as they get more and more familiar with the systems. This might have a negative impact on safety issues because drivers sometimes rely on the systems too much. These behavioral changes based on individual experiences over time are defined as behavioral adaptation (Sullivan et al. 2016).

Behavioral adaptation can be triggered by different factors such as drivers’ reduced risk perception, overtrust in the capability of automated systems, false estimation concerning the systems’ competencies, and potential loss of engagement in the driving task (e.g., Itoh 2012; Rudin-Brown and Parker 2004; Sullivan et al. 2016). Reliance to automated systems even in critical situations where the system might become inactive increases the risk of not being prepared to take over vehicle control in the right moment and with the reaction quality required. For example later braking in case of ACC use compared to manual driving was shown by Nilsson (1995). Larsson et al. (2014) found similar effects in experienced as well as in inexperienced ACC drivers. These results show that slower reaction does not seem to be only a cause of unfamiliarity with the system but might be caused by behavioral adaptation. Given these facts, behavioral adaptation seems to play an important role for the evaluation of traffic safety in the context of automated systems (Mehler et al. 2014).

According to Manser et al. (2013), behavioral adaptation may be structured into three temporal stages: immediate (the first learnings when the driver initially interacts with the system), short term (over days or weeks) and long term (over weeks or months) (Sullivan et al. 2016). While immediate behavioral adaptation of drivers to the use of highly automated systems has already been considered in a number of studies, little is known about the development of these behavioral changes over time, when familiarity with the system increases. To address this issue, a long term static driving simulator study will be conducted in spring 2019, based on the research project TANGO.

1.3 Research Project TANGO and User-Centered Development

The current study is based on the research project TANGO (German abbreviation for Technologie für automatisiertes Fahren nutzergerecht optimiert, English equivalent Technology for autonomous driving, optimized to user needs), funded by the German Federal Ministry of Economic Affairs and Energy, in cooperation with Robert Bosch GmbH, Volkswagen Aktiengesellschaft, MAN Truck & Bus, and University of Stuttgart. The project aims at developing an Attention and Activity Assistance system (AAA; see Fig. 1) which shall enable truck drivers to engage in secondary tasks during automated driving phases. The AAA provides the drivers with diverse non-driving-related activities based on current traffic situation, automation level up to SAE level 3 (SAE international 2018), and the driver’s current attentional state. It is designed by combining proven environment sensors with novel cabin-interior sensors and novel HMI-concepts (projekt-tango-trucks 2017).

Fig. 1.
figure 1

The Attention and Activity Assistance System (AAA)

Full size image

For market success, automated driving systems to enhance traffic safety have to be accepted by the user. This can be accomplished by a positive user experience which can be achieved by a user-centered development (DIN EN ISO 9241-210 2010). That for the AAA is developed in a user-centered way.

User-Centered Development of the AAA.

The user-centered development of the AAA starts with various empirical studies for a deep understanding of the current working situation likes and dislikes of truck drivers, as well as their attitude towards automated driving. Based on the studies’ results, functionalities, interactions and other prototypes of an AAA are realized in an iterative development process: In first iteration, a great number of very easy to realize “paper prototypes” are created and then evaluated in focus groups and online studies in form of user stories. In second iteration, the most promising ideas are provided in simulation and tested in laboratory environment. Finally, the optimized concept is implemented in a truck for evaluation in realistic environment. But how can the construct user experience be described?

Six Facets of User Experience.

User experience is a complex phenomenon. In most situations, only some experiences are selected consciously for decision making, whereas the majority of experiences influences the user acceptance on a subconscious level. For the systematic design of a good user experience of products or services it is important to know as much as possible about the complex phenomenon. For a broad understanding of user experience (Engeln 2013; Engeln and Engeln 2015) published a six facets model (see Fig. 2).

Fig. 2.
figure 2

Six facets of user experience

Full size image

The facets are not selective but influence each other. In particular, the facets task (with interaction) and self-expression are influenced by comprehensibility, convenience, joy, and aesthetics of use.

Task (incl. interaction) addresses the experienced utility of an offer or product. Does it help to reach one’s goals, reduce the effort or improve the quality of the result?

Self-expression means the (mostly implicit) thoughts about what others would think about the user when he owns or uses the offer or product. Does the user identify himself with the product, is he proud of it or rather ashamed?

Learnability runs with the investment of effort for competent use. Is the use intuitive, self-explaining or do I have to learn for it?

Convenience of use is relevant in case of dominant extrinsic motivation, when the user mainly wants to gain the results of an action. A good convenience of use leads to relaxation, a bad one to pressure.

Joy of use becomes important in case of mainly intrinsic motivation, when the action itself dominantly motivates the user. A good joy leads to happiness, a bad one to boredom.

Aesthetics focusses impressions of all senses. People adore good views, sounds, tastes, smells or touches. They avoid negative ones.

Asked for their user experience, people typically talk about the task but seldom about the self-expression. Very often, people stop reflecting when they have found “one good argument”, although their experience in fact is affected by several factors. Because of that, the whole phenomenon needs to be taken into consideration in order to design a good user experience.

This report focuses on a specific long-term simulator study based on TANGO, which looks on changes over time in user experience and behavioral adaptation to automated driving and interaction with the automated system. As the TANGO project also deals with the similarities and differences between truck and car drivers and wants to give insights concerning the transfer between truck and car context, this study is focused on car drivers’ behavioral adaptation.

1.4 Objectives

The central objective of this study is to learn more about the development and change of drivers’ adaptation behaviors in time-critical take-over scenarios with low time budgets which require an immediate reaction by the driver. The main interest is a better understanding of the development or change in behavior over time, when familiarity with the system increases. The concrete question addressed by this goal is: Does take-over behavior in time-critical situations differ between drivers with and without experience in time-critical takeover situations?

Furthermore, the study aims at getting insights into changes in user experience over time as well as in changes of drivers’ safety behavior. Concerning safety behavior, the study will for example look at quality of take-over and control gazes during automated driving. Effects concerning traffic safety will be analyzed following the I-TSA scales developed within the German research consortium INVENT (Böttcher et al. 2005; Glaser et al. 2005).

2 Method

In order to access the development of behavioral adaptation and user experience over time, a long-term static driving simulator experiment will be conducted in spring 2019.

2.1 Driving Simulator and Driving Scenario

For the purpose of the experiment, the driving simulator at Stuttgart Media University (see Fig. 3) will be used. It is based on a platform with multiple interacting high-end computers. The simulation is presented on three large displays in the participant’s main field of view. In addition, we placed two smaller side-displays for showing virtual mirrors of the rear scenery. A realistic seating position is provided by the combination of actual automotive components, i.e., driver seat, steering wheel, and pedals. The equipment is mounted within a steel frame to ensure a consistent environment for all participants. The haptic feedback from the input gear resembles real automotive technology in case of the gear shifter, the pedal stiffness/response, and an actuator behind the steering wheel for active steering feedback controlled by the simulation. Surround audio gear also gives the participant a more holistic experience.

Fig. 3.
figure 3

The driving simulator at Stuttgart, Media University

Full size image

The simulator mockup is placed in a tiny laboratory room without window but ventilation. This ensures a calm and reproducible environment for the experimental situation. The simulation is controlled from a separate control computer outside the laboratory room. The participant is able to communicate with persons outside via speaker.

The SILAB simulation software (WIVW - Würzburg Institute of Traffic Science n.d.) is used. The modular software provides deep configuration capabilities for customizing the simulation process. The configuration of the simulation for this specific experiment uses a single road course with alternating events over the course of the full experiment. As the driving situation is intended to simulate a daily way to work the same route is used for all rides. Participants will start aside the road and drive manually towards a highway entrance. On the highway, participants will be asked to transfer vehicle control to the automated driving system. Within the autonomous driving situation, several take-over requests are placed. Some take-over requests are critical some are uncritical according to the time given to take over control. After the experiment-related events will be completed, participants will drive manually off the highway and park the car.

On runtime, the simulation measures several driving related data. The following list contains the most relevant of them:

  • Ego-car speed and acceleration (longitudinal and lateral)

  • Other vehicle speeds and accelerations (longitudinal and lateral)

  • Distances between the ego car and other vehicles or objects

  • Runtime of the simulation

  • Position of ego-car and other cars (absolute and lane-dependent)

  • Inputs of the participant, i.e. steering angle, pedal offset, gear shifter

Collecting these metrics enables to calculate further simulation related values like time to collision with other vehicles as part of the scientific questions behind the experiment and for analyzing effects on traffic safety.

2.2 Experimental Design

A 5 × 2 mixed design with factors driving session (within) and experience with timecritical take-over situations (between) will be used. The factor driving session has five levels (ride1/ride 2/…/ride 5). Rides should take place once a week for five weeks à 45 min each ride. The second factor (experience with time-critical takeover situations) has two levels (time-critical situation: yes/no). The experimental group will experience time-critical take-over situations in rides 1 to 4 whereas a control group will only experience non-time-critical take-over situations in rides 1 to 4. In ride 5, both groups will experience a time-critical take-over situation.

2.3 Procedure

The study will be conducted in the static driving simulator (see Sect. 2.1). For first trial, participants will be welcomed in the laboratory and will get written as well as oral instructions concerning the experiment. Participants will be instructed that the experiment aims at evaluating the user experience of secondary tasks under realistic conditions. They will be asked to imagine being on their way to work while driving. Furthermore, participants will be asked to complete a demographic survey and a data protection declaration.

After filling in all documents, participants will start with a training session of about five to ten minutes to get familiar with the simulator and to exclude participants who suffer from motion sickness (e.g., Brooks et al. 2010). For this training session a drive through a small piece of landscape including curvy land roads and a part of a small town is chosen. During the training session, participants will learn how to activate the autonomous driving and how to take over vehicle control. Before starting the test ride they will be told that they can stop the experiment anytime they do not feel comfortable anymore.

After the training session, the eye-tracking system will be applied and calibrated. The ride will start with a manual driving session for about three minutes. Participants will be instructed not to overtake during this session. After three minutes, participants will be asked to start automation. This is done by pressing a button located to the right of the steering wheel. During the automated driving session, the ego car follows a lead vehicle without performing takeover maneuvers. This enables a better control of events within the simulation scenario for example surrounding traffic.

Participants in both groups will be asked to perform a secondary Sudoku task on a tablet during the automated driving session. After 20 min of autonomous driving, participants of both groups will be faced with a take-over situation. While control group participants will experience a non-time-critical take-over request because of road works, experimental group participants will be faced either with a time-critical or a non-time-critical request. The time budget in the time-critical take-over situation will be seven seconds (according to Gold et al. 2016) the budget for the time-uncritical take-over requests will be 60 s. Participants will be instructed to take-over vehicle control immediately. The following events are chosen to define the time-critical take-over situation:

  • Deer crossing

  • Vehicle going into lane from the right side

  • Unsecured accident site

  • Sudden emergency braking of lead vehicle because of traffic jam

Control group participants and those of the experimental group who will be faced with the non-time-critical request do not have to take-over immediately. They will be informed that they will have to resume vehicle guidance in one minute because of road works. There will be a reminder 30 s before they should take over. In case participants do not take over, there will be a safety stop.

After take-over, participants will drive manually for about two minutes. Then participants will be asked to give the vehicle control back to the system. Drivers then will drive autonomously for another 18 min. After 18 min, control group participants will experience another non-time-critical take-over request because of an exit ramp whereas experimental group participants will experience a time-critical or a non-time-critical request, depending on the criticality of the previously experienced request: Participants who already experienced a time-critical request will be faced with a non-time-critical request whereas participants who firstly experienced a non-time-critical request will be faced with a critical one. This counterbalance is done to avoid learning effects due to the order of the time-critical take-over situation. After the take-over, participants will be asked to drive manually until the next highway exit. Participants will drive to a specific parking space and will stop the car there. The procedure of the original experimental phase is graphically displayed in Fig. 4 separately for experimental and control group.

Fig. 4.
figure 4

Procedure

Full size image

2.4 Participants

50 students with normal or corrected-to-normal sight and hearing and a valid driving license will be acquired. Participants will be paid 10 Euros per test trial and a bonus of 40 Euros as far as they will have taken part in all five repeated measurements. Participants will be randomly assigned to one of the two between subject conditions (experience with time-critical take-over situations vs. experience with non-time-critical takeover situations).

2.5 Measurements

Besides the simulation-based metrics, biometrics from the participant will be collected. The main biometrics here will be gaze-based interactions of the participant with the simulation and with objects aside the simulation (i.e. secondary task). For that purpose, a mobile eye tracking system will be used.

Eyetracking.

For the assessment of the participants’ glances behavior, the Dikablis Professional Glasses 3 (Ergoneers. Hard- und Software für Verhaltensforschung & Eye-Tracking n.d.) and the associated software D-Lab will be used. The binocular head-mounted eye-tracking system comprises an adjustable scene camera capturing the wearer’s visual field with an aperture angle of about 90 and a resolution of 1920 × 1080 pixels. The two adjustable infra-red eye cameras, attached to a leg underneath the eyes, have a resolution of 648 × 488 pixels with a tracking frequency of 60 Hz. The cameras’ housings contain small infra-red LEDs illuminating the eyes. The images of the illuminated eyes are digitally processed frame-wisely in order to detect the pupils with an accuracy of 0.05° visual angle. The glance direction is estimated with an accuracy of 0.1° to 0.3°.

The Dikablis glasses will be used wirelessly with the data being stored in a small transmission unit and sent via WiFi to the recording computer, a Dell Precision 7520 with the recording and analysis software D-Lab installed. In order to enable automatic analysis of glance behavior for different areas of interest, so-called markers, graphical black and white patterns of size 6 × 6 and 10 × 10 cm, will be arranged in the simulator. The markers are based on simple high contrast visuals (mostly similar to QR-code) and are placed in strategic positions on the main displays or on other objects. The analysis software D-Lab detects these reference points in the recording environment via image processing and determines the location of marker-coupled visual targets despite head movements. Three markers will be displayed directly in the middle screen corners of the simulation to tag the road. The speedometer unit will be coupled to one marker displayed left to the instrument cluster. Exterior mirror markers will be attached below the respective mirror screens and one marker will be displayed on the tablet which is used for the secondary task (Sudoku).

Behavior.

To get insights into changes of behavioral adaptation over time, take-over behavior, reaction times and -correctness as well as driving behavior after take-over will be measured.

Stress/Demand.

Subjective measurements will also been taken into account to get information concerning participants’ stress and user experience over time. Stress and demand over time will be measured with the Rating Scale for Mental Effort (RSME, Eilers et al. 1986) whereas user experience will be measured with a pilot version of our newly developed UX questionnaire.

2.6 Analysis

Data will be analyzed with the statistic software SPSS and custom-designed software by Blickshift.

3 Results (Preview)

As the study is going to start in March 2019, the results cannot be presented at this point. It is planned to complete the collection of data by July, 2019. In parallel to the conduction of the study, data will be progressively preprocessed and analyzed so that at the HCII, first prospects of the change of driver behavior and attitudes over time will be presented.

4 Conclusion

The planning and execution of a longitudinal study with data collection over several weeks needs a high amount of investment. The advantage – compared to classical single data collection – is the knowledge about participants’ behavior and experience development over time. In contrast to a very first contact to automated driving, this longitudinal research design will come somewhat closer to the reality of automated driving: When people own an automated driving system, they get used to it very quickly and change their usage behavior and experience within a few days (Arnon et al. 2014). Therefore, it is reasonable to assume that the planned experiment will reveal behavioral changes in drivers during the interaction with the automated system.