Trust in Automated Vehicles

2.1 Trust and Trust in Technology

Trust is a complex concept, often understood as either a type of interpersonal relationship or personal disposition towards others or contexts inhabited by other individuals [31], [2], [27]. The disposition or attitude can be understood as an expectation that certain conditions are fulfilled or goals are reached [19]. Such a relation or disposition of trust does not, however, necessarily involve only human individuals. Trust occurs between two types of agents, the trustor – the one who trusts – and the trustee – the one who is trusted [6]. When one of these agents, likely the trustee, is a technological artifact or a system, then this is referred to as “user-system trust” or “trust in technology”, as opposed to interpersonal trust.

There are several definitions of ‘trust’ in computational and technology contexts. Hoff and Bashir [17] state that most explanations of trust consist of three different components:

1. A set of trustor, trustee, and something that is at stake

2. An incentive for the trustee to perform the task

3. The possibility for the trustee to fail the task.

2.2 Technology Acceptance and Trust

The question of whether a technology or system is accepted and integrated into their user’s daily life has to be asked anew with every new technology or system as they are introduced into society. In research, the Technology Acceptance Model (TAM) by Davis [10], has become a widely used key model to predict the potential acceptance or rejection of a technology or system. The model kept on developing throughout the years and experienced various extensions. Trust was added as one contributing factor to the technology acceptance model by various researchers [20]. Based on the TAM, Constantinides et al. [8] propose a new model of technology acceptance that also integrates the degree of autonomy to predict the acceptance of Internet of Things (IoT) retail services. Similar to ADAS in the automotive domain, IoT technologies are ubiquitous, intelligent and autonomous. The results emphasize that the customer acceptance of IoT services decreases when technological autonomy grows. Further, they found positive direct effects of technology trust on intention to accept the IoT services. Technology trust even gained relevance in situations when technological autonomy was high. These findings highlight that user perceptions of trust are important, especially when technologies are highly autonomous. Thus, both trust and degree of autonomy can be considered constituting factors for technology acceptance of automated systems.

2.3 Automated Vehicles and Trust

Vehicles are not just classified in a binary manner depending on whether they are automated or not but regarding their degree of automation, as there are many different ways in which a vehicle can have assistive or automated functionalities. The most well-known classification of automated vehicles is provided in the SAE J3016 standard [32]. This standard provides a descriptive categorization of vehicles on the basis of their technical capabilities. Generally speaking, the higher the vehicle’s capabilities are, i. e., the more tasks it is capable of performing, the higher its level of automation. Thereby, capabilities include the safe operation of the vehicle (i. e., latitudinal and longitudinal control), as well as the task to monitor the environment.

The SAE scale defines six levels, ranging from Level 0 (no driving automation) to Level 5 (full driving automation). See Table 1 for a brief description of the levels.

These levels indicate, on one hand, that it will be essential for the automated vehicle to communicate its driving state and, thus, what is expected from the driver. On the other hand, it shows that there is more than one task the driver or the system have to perform (e. g., steering, accelerating, breaking, monitor the environment, supervising the system), each of which translates into different requirements for either the driver or system.

Numerous considerations will affect the development, success, and adoption of automated vehicles in the near future. Trust, in close relation to technology failure, is one of the most important ones to consider on the drivers’ side. In order to prepare the future drivers for the realities of highly automated driving, the first part of that process must be “building trust and proficiency in today’s ADAS, and charting a vision toward the policies, technologies, and human-centered interactions that will support tomorrow’s driverless vehicles.” [28].

Several researchers have already investigated trust in the context of automated vehicles, but results are inconclusive. While Gold et al. [15], for example, found that the experience of a drive with a highly automated driving system increased the drivers’ self-reported trust in automated vehicles, Feldhütter et al. [13] did not discover any such effects. Also, almost all studies on trust in automated vehicles are usually conducted in driving simulator environments. However, the level of trust experienced in a real automated vehicle may differ significantly from the level of trust experienced in the safe surroundings of a driving simulator, since simulations do not entail the risk necessary for “true” trust relations [9], [24].

In order to shed some light onto aspects that are relevant for understanding trust in the context of autonomous vehicles, we present results from trust research in the field of dynamic web services. In the following chapter, we first outline what the domains of dynamic web services and automated vehicles have in common together with a brief description of the project, and then describe a selection of results that can be considered relevant for both domains.

3.1 Trust vs. Trustworthiness

Trust and trustworthiness are on opposite sides of the same coin and this became more and more visible as the platform development progressed. The term ‘trustworthiness’ refers to characteristics of someone or something that is the object of trust [9]. When it comes to the perception of trustworthiness of websites, several factors play a role, such as ease of navigation or freedom from typographical errors (see [3], [14], [9], [11]). Also, internet users may have different attitudes regarding privacy, leading to a different perception of the trustworthiness of websites. According to Ackerman et al. [1], Internet users may be either privacy fundamentalists, pragmatists, or marginally concerned. Riegelsberger and Sasse [30], [29] point out that the assessment of the trustworthiness of a website is always a secondary task to the primary goal of the website user. This is an important point, since it emphasizes the main purpose of automated systems, i. e., to let users focus on their primary task. And it also means that trustworthiness cues can not take the user’s center of attention most of the time, as that would distract them from their primary task.

Within the project, we found that as soon as website users had to put more effort to trust assessment than to their initial primary goal, the automated systems lost one of its main benefits, i. e., the added convenience due to automation. Thus, establishing trust and providing adequate security is a challenging balancing act, as inadequate trustbuilding strategies and/or feedback solutions may result in a decreased user experience due to a perceived loss of the advantages of the automated system. Based on our findings, we concluded that “trust assessment is a secondary task of primary importance” [22]. As a rule of thumb, trustworthiness cues must always be second to the interface means necessary for executing the primary task, whichever that might be at the time. If trustbuilding is elevated to the primary task level, it runs the danger of serving nothing but its own purpose, decreasing interaction quality and being detrimental to trust in the long run.

3.2 Trustor and Trustee, or Who Trusts Whom?

In our studies conducted for the ANIKETOS project, we found that users generally had a low initial trust in dynamic web service platforms. One explanation was that such services are not yet common and, hence, users lacked experience with them. They did not know what they could or could not do and had no interaction experience that could have given them any previous insight on how secure or trustworthy they might be. This may be similar to automated vehicles; the lower the initial trust can be expected, the more novel and, therefore, less known the technology is to the average user.

Furthermore, in the context of dynamic web services, we had also found that it was important for users to know who they were interacting with in terms of ensuring security in the background. In other words, it was not always clear to them who they were in a trust relationship with. We called this phenomenon the “opaqueness” of the trustee [22]. The degree to which the trustee is opaque to the trustor also influences how much the trustee is trusted, with more information available usually leading to higher trust.

Opaqueness can occur on two levels: identification or feature level. On the identification level, it is difficult for the trustor to simply identify a trustee or among several potential trustees. This can be postulated to mostly cause reduced dispositional trust, as the directed trust relation is unclear. On the feature level, the identity of the trustee may be clear but it is unclear what their exact features are. Knowing these features helps the trustor decide what the trustee is capable of performing, which influences their trust in the trustee to successfully perform certain tasks. Opaqueness on the feature level does, therefore, also influence directed trust.

3.3 Opening the Black Box – When and Why

An automated system could actually remain a black box as long as there is no risk involved for the user. However, as soon as risk is involved, a black box will become problematic. If the user is unaware of what is happening or what the system is doing, it is likely that their uncertainty will increase and their trust in the system will decrease accordingly [12].

We found that the involved risk for the website users had a strong influence on their desire for feedback. That is, if there was the risk of financial loss due to an insecure web service, or the danger to lose personal data, it was also of high importance to be informed by all means. Users would care less if there was, e. g., a change of weather service. As outlined above, information about the underlying system handling the web service recomposition was also of importance. In our studies, participants suggested that information on its reputation or a “trust seal” could help to make this black box more transparent. Another issue we identified was that users might not always perceive a potential risk as such. Hence, provided feedback should also include information on the type of risk involved.

Regarding feedback density, the challenge is to provide the users with all information necessary without annoying or scaring them. We found that particularly in situations with higher risk involved, users need to be informed and warned so that they have the chance to react immediately to potential security threats. Thereby, it is important that the user clearly understands what the issue is and what information needs to be provided in an appropriate manner. Of course, what is considered appropriate depends on the frequency and how much effort of the user a response to the feedback entails.

In analogy to driving a highly automated vehicle, using a dynamic web platform means that the user is leaving a lot of the control and choices to the system. In the ANIKETOS project, we found that it is important that users get a choice in case an unexpected event happens of which they get informed of and which involves risk for them. For example, if a web service for online payment is insecure and users are notified, they also want to have a choice of how to proceed instead of the system replacing the web service for a secure one automatically. That is, if users have no control in such situations, they also will not trust the platform. We concluded that “the user needs to be able to make an active choice right as the feedback happens” [22].

4.1 Communicating Trustworthiness and Trust Calibration

Since trustworthiness cues should come second to the primary task, it is important to first identify what the primary task in a given situation actually is. In the context of automated driving, this is not always clear-cut: the user’s primary task could either be the manual driving task itself or an entirely different task, depending on the vehicle’s automation capabilities and whether the automation systems are active at the time. This is only an issue, however, if we assume that trust cues are always unrelated to – and, therefore, potentially distracting from – the primary in-vehicle task(s).

However, this need not necessarily be the case. Trust calibration means to provide trust cues that influence trust in both a positive (to avoid undertrust) and a negative way (to avoid overtrust). In the context of automated vehicles, such cues would relate to the vehicle’s capabilities and which tasks, maneuvers, or actions it can or can not perform – either in general or in a particular situation [23]. These can be realized, e. g., within the regular user interface communicating system status of the automation system. Instead of simply communicating whether automation is on or off, a display can contain further information on recommended, possible, and impossible actions via a simple color coding scheme, which is one of the more effective ways to communicate trustworthiness cues [22]. Such an indicator does not change the vehicle’s capabilities in any way but instead communicates them more clearly to the driver and helps them adjust their expectations appropriately. An example for such a trust cue communication interface would be a visual sensor status indicator with a color coded output that employs the traffic light metaphor (red, yellow, and green). The interface communicates the confidence in the reliability of its own sensor data at all times with the appropriate cue (green: full confidence – inaccuracies unlikely; yellow: medium confidence – inaccuracies possible; red: low confidence – inaccuracies highly likely). Each cue informs the driver and, in doing so, enables them to adjust their expectations in the vehicle’s performance and their own monitoring behaviour. Note that this is only an illustrative example, which targets only one trust-relevant factor (sensor reliability in this case).

Regarding the potential concern that situational distrust might decrease overall trust, there is a relation to the topic of system transparency and visibility of system status [18], which affects how much trust a person puts in a system. Furthermore, it can be assumed that repeated and successful interactions with the vehicle and its different functions would result in higher reliance. These successful interactions are more likely to occur, the better the user interface is able to calibrate trust and guide the driver’s interaction. We concluded that “reliance is fundamental for trust and reliance-fostering measures should be treated as important as (or perhaps even more important than) traditional trust building strategies” [22].

4.2 The Value of Interventions

The strong push towards full automation is, as initially mentioned, well motivated. However, beyond technical developments regarding motivation taking time to mature, there might be an additional reason to extend the transition phase. That reason is the potential of intervention capabilities to positively influence trust in risky contexts. In the web context, it seems almost trivial to state that higher risk equals higher need for information and intervention possibilities. This is because there is a wide variety of interactions on the web ranging from practically zero to high risk (e. g., merely navigating to a website vs. performing a monetary transaction). In the automotive context, however, things are a bit different, as the act of driving a vehicle is inherently risky. An argument can be made that this is not entirely different for on-line interactions, as even visiting a website can be harmful if that website had malicious code injected into it.

Operating a vehicle, however, always brings with it the risk of bodily harm or even death the moment one sits in the driver’s seat, making it an especially (and permanently) risky context. In combination with the findings described in section 3.3, this means that there is essentially no moment in which the automation should be a complete black box to the user. The question is then not if the black box should be opened or not, but how far it should be opened. As we explained in Sections 3.1 and 4.1, any such opening of the black box can interfere with the driver’s primary task – unless the information provided is directly relevant to said task.

One possible logical consequence from this combination of factors is to assume intervention capabilities by the driver to be an important trustbuilding measure. This is most easily realized in SAE level 3 vehicles, where intervention readiness is still an essential requirement on the driver’s part. Thus, extending instead of reducing the transition time to full automation might actually help instead of hinder trustbuilding in automated vehicles. This provides an interesting alternative perspective to the current “rush towards level 5”. Instead of trying to get there as quickly as possible, there might be additional trust-related advantages to giving both technology and infrastructure the time they need to mature. It should be noted, however, that level 3 is not the only automation level foreseeing interventions. A level 4 vehicle, which reaches the end of its operational design domain (ODD), also requires intervention or executes a minimal risk maneuver if no intervention occurs. The SAE J3016 standard includes intervention requests even for level 5 vehicles [32] but defines them as not mandatory to respond to. Thus, there is an argument for exploring the potential of not only critical but also non-critical interventions across driving levels.

4.3 Knowing Who to Trust Means Knowing Who is in Control

We found that in a dynamic environment, it can be difficult to clearly identify the trustee, which makes it difficult to properly adjust one’s expectations. This causes an unclear or even nonexistant trust relationship. In the automated driving context, this situation is rather similar, although for different reasons. What it comes down to are the different levels of automation and whether the human driver or automation system are in charge of the vehicle’s controls. Depending on who is in control and whether or not this is visible or communicated to other road users, the trust relation can quickly become unclear even when only two actors are involved.

Figure 3

The trustee-relationship with a manually driving human trustor and vehicles of different automation levels.

In Figure 3, we illustrate three simple scenarios to illustrate the opaque trustee phenomenon for automated vehicles. Figure 3 A shows a level 0 vehicle approaching an intersection with another level 0 vehicle approaching from the right, indicating the intention to turn left. The driver going straight will need to trust the other vehicle to give them the right of way, which they can verify by looking at the driver’s head and eye movements. The trustee, in this case, is clearly the human. In B, the same level 0 vehicle encounters a level 5 vehicle signaling to turn left. The trustee, in this case, is clearly the automated vehicle. However, the level 0 driver can only establish the proper relation to the system, if they know that the vehicle is fully automated, either by the driver seat being empty or the person in the driver seat being visibly occupied with a non-driving task. But even that is not a safe indicator, as the individual in the other vehicle might simply be a careless driver of a vehicle of a lower automation level. Figure 3 C shows an even more difficult scenario, in which the level 0 driver encounters a level 3 driver. The trustee is not defined by the automation level in this case but by whether the automation is on (trustee: system) or off (trustee: human). Furthermore, level 3 requires the driver to respond to intervention requests (fallback performance), so even when the automation is active, a properly adjusted trust relation must assign some trust to the human to respond in case of an emergency. The intervention readiness requirement also means that the driver seat can never be empty, making it even more difficult to establish the proper trust relation(s).

This example is an oversimplification that assumes that there will be no other indicators apart from driver position and behaviour to communicate the automation level and who is in control of the vehicle. It is unlikely that this extreme will be the case and there can be expected to be at least static indicators of automation capabilities (e. g., differently colored license plates). What the example is supposed to illustrate, however, is that even such static indicators are likely not enough to eliminate critical ambiguities. Control modes can change dynamically, making, e. g., the information that a vehicle is a level 3 vehicle alone not enough to establish the proper trust relations. This is not limited only to level 3; control changes can occur in level 4 vehicles as well (upon reaching their ODD, as mentioned in section 4.2) and, at least per definition, even in level 5 vehicles on a non-mandatory basis. It will, therefore, be important to design for proper communication of control modes not only between driver and vehicle but also to outside the vehicle, either on a visual basis for manual drivers or on a vehicle-to-vehicle basis for connected vehicles.