Keywords

1 Introduction

Partial-autonomous vehicles are among us and represent a prominent testing ground for assessing the human interaction with autonomous vehicles. The Society for Automotive Engineers [1] defines five levels of driving automation based upon the system’s capability to execute lateral and longitudinal maneuvers, monitor the driving environment, respond to emergencies and drive without the aid of the human driver in various traffic scenarios. Whilst we expect vehicles with level-3 to level-5 autonomous capabilities to gradually hit the market in the next 2 to 25 years [2], level-2 vehicles – i.e., vehicles equipped with systems capable of executing steering and acceleration operations but requiring the human driver to monitor the traffic environment, are currently being driven on US roads.

Over the last fifteen years, the volume of studies investigating public’s attitudes toward autonomous systems has grown exponentially. Topics such as trust and acceptance toward vehicles with autonomous capabilities have been the focus of investigation in the driving community. In the study by Koo et al. [4] authors manipulated the type of information provided to the driver by a collision avoidance system and investigated its effect on trust. Whenever a possible collision was detected, the collision avoidance system automatically applied the brakes and one of four possible warning messages was presented to the driver. The four warning messages contained information regarding: the behavior of the system (i.e., “car is braking”; How message), the reason for the system to intervene (i.e., “Obstacle ahead”; Why message), a combination of the two (i.e., “Car is braking due to obstacle ahead.”; How + Why message), or none of the above (no message). Results showed that the content of the message had a significant effect on ratings with participants surprisingly feeling less positive about the system in the How + Why compared to the remaining three conditions. Interestingly, highest ratings of trust were found in the Why condition where the information provided to the drivers was more salient and, thus, quicker to process in the context of near collisions. More recently, Itoh et al., [3] investigated participants’ interaction with a simulated semi-autonomous vehicle equipped with an auto braking system. Results showed that participants positively evaluated the assistance system with respect to its value is avoiding collisions. In particular, feelings of acceptance increased in high emergency scenarios.

One possible limitation of the studies investigating would-be users’ attitude toward partial to full autonomous driving stems from their indirect experience with such technology. In many studies information regarding user interaction with partially to fully autonomous systems was collected either in simulated scenarios [5] or via surveys [6, 8]. In this study, we investigated user interaction with partially-autonomous vehicles and observed how direct exposure to level-2 vehicles shaped the user experience of such systems. Further, given the limited availability of highly and fully autonomous vehicles, testing a level-2 vehicle on the open road will allow to identify challenges characterizing the user interaction with partial automation and, thus, develop possible solutions for level-4 and 5 vehicles.

The aim of this study is to investigate how naïve drivers interact with level-2 vehicles and, in particular, assistance systems such as Adaptive Cruise Control and Lane Keeping Assist. In this study, participants drive a partial autonomous vehicle on highway roads for one hour and interact with ACC and LKAS using the human-machine interface (HMI) available on the 2016 Honda Accord equipped Honda Sensing. Subjective ratings of trust and acceptance as well as comments regarding the overall interaction with the systems are collected.

2 Method

2.1 Participants

Ten participants (one male) participated in this study. Participants had an average age of 25 years (standard deviation: 3 years). All participants had normal neurological functioning, normal or corrected-to-normal visual acuity, normal color vision, a valid driver’s license, and were fluent in English. Participants admitted they did not have any prior direct experience driving vehicles with ACC and LKAS. A University of Utah Institutional Review Board (IRB) authorization.

2.2 Materials

Participants drove a 2016 Honda Accord equipped with Honda Sensing. Honda Sensing is a suite of advanced driver assistance systems (ADAS) including: Collision Mitigation Braking System – an automatic braking system that applies the brakes whenever a collision is deemed unavoidable, Adaptive Cruise Control, Lane Keeping Assist System, Road Departure Mitigation – a system designed to help steer to help keep the vehicle from leaving the road, Lane Departure Warning – a system designed to monitor vehicle lane position and alert when it drifts into a new lane without the driver signaling, Forward Collision Warning, - a system integrated with Collision Mitigation that is designed to detect the presence of vehicles in front and alerts the driver when approaching at high speed (see Fig. 1).

Fig. 1.
figure 1

2016 Honda accord XL with Honda sensing used in this study

Full size image

SAE describes level-2 vehicles as following: “the driving mode-specific execution by one or more driver assistance systems of both steering and acceleration/deceleration using information about the driving environment and with the expectation that the human driver performs all remaining aspects of the dynamic driving task” (see Fig. 2). The 2016 Honda Accord equipped with Honda Sensing thus qualifies as SAE level-2 vehicle.

Fig. 2.
figure 2

Levels of autonomy (Society for Automotive Engineers (2014))

Full size image

A GoPro camera was used during the drive for recording the dialogue inside the cabin. The questionnaire (Questionnaire 1) used to collect subjective ratings was composed by 10-point Likert scales measuring: overall opinion toward ACC and LKAS, trust, acceptance, usability, ease-of-use, stress, feeling of safety.

2.3 Procedure and Design

Before the study began, participants filled out a consent form approved by the University of Utah Institutional Review Board. Once in the vehicle, participants provided information regarding their demographics and knowledge of autonomous vehicles. We adopted a pre-post experimental design with participants completing questionnaire 1 before and after driving the level-2 vehicle and experiencing ACC and LKAS. This allowed us to measure how their knowledge and attitude toward ACC and LKAS changed over time with experience of the systems. Before driving, participants completed questionnaire 1 for the first time (pre). After completing the questionnaire, a general overview of the commands to be used for controlling ACC and LKAS was provided to participants by the research assistant sit next to them. We did so to help participants familiarize with the new vehicle and HMI of ACC and LKAS. The starting point for the drive was in the parking lot of the Department of Psychology at the University of Utah. After 15-min drive in a residential area of Salt Lake City, Utah, participants entered the eastbound I-80 highway and drove on the southbound I-215 highway for a total of 40 min (see Fig. 3 for details about the route).

Fig. 3.
figure 3

Route considered in this study.

Full size image

During the first 10 min of the drive, participants were left free to familiarize with the HMI of ACC and LKAS. After the 10-minute familiarization phase, participants were instructed to follow a lead vehicle at a constant distance using ACC. The lead vehicle operated by a second research assistant drove in the middle of the three lanes and varied its speed between 60 mph and 70 mph every 5 min. During the drive, participants were encouraged to engage both ACC and LKAS while keep monitoring the traffic environment and maintaining a safe driving. The thinking-aloud technique was used to record information regarding driver interaction with the vehicle HMI. The research assistant driving in the test vehicle with participants prompted them throughout the drive to provide insights, comments, opinions about interacting with ACC and LKAS using the commands available on the steering wheel and display available in the instrumented cluster. Such information was then transcribed for further analysis. At the end of the drive, participants completed questionnaire 1 (post) and their opinions and subject ratings regarding ACC and LKAS were recorded.

3 Results

Paired-sample t-tests were conducted to investigate whether subjective ratings varied over time. For ACC, significant pre-post differences were found for trust, t(9) = 2.6, p < .05, and ease-of-use, t(9) = 3.3, p < .05, scales with participants expressing higher ratings of trust and ease-of-use in the post questionnaire compared to the pre questionnaire. No significant pre-post differences were found for opinion, feeling of safety, acceptance, stress (see Fig. 4).

Fig. 4.
figure 4

Subjective ratings for ACC recorded in pre-post questionnaires. Error bars represent standard errors.

Full size image

For LKAS, significant differences between pre-post questionnaires were found for the stress scale, t(9) = 2.7, p < .05, with ratings of stress increasing over time. No significant differences were found for opinion, trust, acceptance, usability, ease-of-use, feeling of safety. Data are presented in Fig. 5.

Fig. 5.
figure 5

Subjective ratings for LKAS recorded in pre-post questionnaires. Error bars represent standard errors.

Full size image

Participants’ opinions and comments toward ACC and LKAS collected via the thinking aloud technique were transcribed. A list of selected comments is presented in Table 1.

Table 1. Selected list of comments
Full size table

Sentiment analysis [9, 10] was run on the transcribed comments to investigate users’ affective state toward ACC and LKAS. The RSentiment package (version 2.1.1 [11]) was used for this analysis. Sentiment indexes were calculated and presented in Table 2. Positive indexes are indicative of more positive affective states whilst negative indexes are indicative of more negative affective states.

Table 2. Sentiment indexes calculated for LKAS and ACC. Average and standard deviations are presented.
Full size table

4 Conclusion

User trust and acceptance represent aspects of primary importance to be accounted for in the development of human-machine interfaces for autonomous vehicles [7]. One main limitation of the studies investigating would-be users’ attitude toward partial to full autonomous driving stems from their indirect experience with such technology. To investigate user interaction with partially autonomous vehicles, in this exploratory study 10 naïve participants drove a SAE level-2 vehicle while engaging Adaptive Cruise Control and Lane Keeping Assist Systems.

Compared to the ratings recorded in the pre-questionnaire, results show that ratings of trust and ease-of-use toward ACC increased over time. This suggests that participants with no prior experience with Adaptive Cruise Control trusted this system more and found it easier-to-use over time. Such findings are in agreement with the study of Kazi et al. [8] in which participants drove a simulated vehicle equipped with ACC with different levels of reliability: 0% (ACC completely unreliable), 50% (ACC reliable half of the times), 100% (ACC always reliable). As the system became more reliable, ratings of trust increased as a consequence and reached ceiling levels after 7 days of exposure to the system.

Different results were found for LKAS. A significant difference in the ratings of stress was found in this study, with ratings of perceived stress increasing in the post-questionnaire compared to those in the pre-questionnaire. This suggests that engaging and interacting with LKAS during highway driving was perceived as a more difficult task compared to interacting with ACC. Such pattern of results is supported by sentiment analysis data suggesting that participants tended to use words with more positive connotations to describe their interaction with ACC compared to those used for LKAS. In particular, participants did not like when LKAS maintained the vehicle in a position within the lane different from that they would maintain during highway driving. Further, some participants suggested that they could possibly trust the system more at slower speeds. Such difference in results between LKAS and ACC may be explained by the fact that, although participants did not have prior experience with ACC, all of them had prior exposure to more archaic speed control systems, i.e., standard cruise control. This might have caused them to be more accepting of ACC and, thus, more skeptical toward LKAS, a system they were mostly unfamiliar with.

This study is of the main importance for addressing future challenges with highly and fully-autonomous vehicles. Results showed that participants, although being trustful of adaptive speed control systems, found interacting with lane maintenance systems a stressful task, especially given the difference between their “driving style” and that operated by the system. Future research is therefore needed to design adaptive, collaborative assistance systems capable of operating the vehicle in ways that are easier for users to trust.