Keywords

1 Introduction

There is an increasing reliance on autonomous agents to perform functions previously done by human actors. Agents are being used in business to maintain interactivity between businesses and their customers [1]; in the U.S. military to conduct dangerous activities such as explosive disposal and firefighting [2]; and in Homeland Security to help analysts process vast quantities of intelligence information [3]. As technology improves, so too does the need to understand how agents can be implemented in a way that yields the most effective partnership between humans and technology [4].

1.1 Agents and Their Roles in Human-Agent Teams

The progression of technology has led to circumstances where complex tasks can be delegated to a machine, automated, and thus be done with fewer errors and by fewer people [5, 6]. One means of implementing automation is through an agent. Agents are hardware or software-based computer system that are characterized by four properties: autonomy, (the ability to operate without direct human intervention for a significant length of time); social ability, (the capacity to interact with humans or other agents using language); reactivity, (the ability to perceive the environment and react to it); and proactiveness, (ability to exhibit goal directed behavior in anticipation of future events) [6–9]. In this manuscript, references to ‘agents’ will use this definition. Robots, by definition, are a kind of agent that are physically embodied [2, 10]. In systems where humans work with agents as team members rather than operators, agents usually play one of three roles: individual support, team support, or team member [9, 10]. When agents support individual team members, they can either be person-specific or task-specific [9]. When agents support a team as a whole, they act to facilitate the group’s teamwork [10]. When agents assume the role of an equal team member, they are expected to act similarly to their human teammates [9, 10]. Thus, similar to a human team member, an agent in a human-agent team must also communicate relevant information to their teammate to maintain shared knowledge and shared intent [9].

In many circumstances, the introduction of automation has changed the human operator’s task to that of monitor and backup [5, 11]. Autonomous agents and their ability to choose goals and act independently also require coordination and cooperation [12]. In systems with flexible automation, agents act in tandem with humans to make decisions, mirroring the relationship between a human and a subordinate [2]. The flexibility gained by this joint decision-making requires continuous collaboration and communication [2]. Human teams have the advantage of flexible communication strategies. Team members elicit relevant information from other team members, which in turn supports development of effective shared mental models [13]. However, in human-agent teams, the agent team members often cannot effectively share information without some means of translating their ‘understanding’ [3, 14]. By establishing a common understanding of the situation, the task, the team members and their respective duties, human teams are able to coordinate effectively [13, 15]. Similarly, effective human-agent teams also maintain a shared understanding of the situation and of their teammates [9]. When both parties maintain a shared understanding of the team environment, they can give, seek, and receive clarifying feedback, which are critical actions for teamwork.

To support the shared awareness and intent needed to perform as an effective human-agent team member, the agent must share information pertaining to its historical and current operation, how the underlying algorithms of the agent govern its behavior (the agent’s “reasoning”), and the extent to which it acts in accordance with the designer’s intent and the operator’s goals [2, 16, 17]. Agents that communicate their performance abilities, intent, reasoning process, and future plans in a way that facilitates operators’ comprehension of such provide transparency [18, 19].

2 Transparency

2.1 Transparency Overview

In the context of human-agent interaction, transparency has been described as a method by which a human and a machine can gain shared intent and awareness [19]. A transparent system facilitates this understanding by explaining its choices and behaviors, allowing its human operators to understand the way it works [20]. One approach, the Belief, Desire, and Intention (BDI) view describes the agent as having mental attitudes [11, 21]. The BDI approach to transparency, communicating the information, motivational, and deliberative states of the agent, helped inspire the Situation awareness-based Agent Transparency (SAT) model [18, 21]. The SAT model informs the design of transparent systems by supporting the human operator’s situation awareness [18]. In the SAT model, a transparent system communicates three levels of information [18]:

  • Level 1 describes the agent’s current actions and plans and its knowledge of the environment,

  • Level 2 describes the agent’s underlying reasoning behind its actions and plans, and

  • Level 3 describes the agent’s predictions about its future state or outcomes of its planned actions

Given the complexities of the transparency construct and the information needed to support it, Lyons divides transparency into two categories, Robot-to-Human transparency and Robot-of-Human transparency, a useful demarcation when describing humans and autonomous agents as teammates [19].

2.2 Models of Transparency

Robot-to-Human Transparency describes models of transparency which focus on the agent’s information about the world [19]. The following models are in this category: the intentional model, the task model, the analytical model, and the environment model [19]. The intentional model focuses on communicating the purpose of the system through the use of social intent cues; the task model focuses on communicating information pertaining to the agent’s task, expressing its goals and its progress towards meeting those goals, its capabilities, and its performance while pursuing those goals; the analytical model focuses on the underlying principles the agent uses to make decisions; and the environment model focuses on communicating variance in terrain, weather, and temporal constraints to humans [17, 19].

Robot-of-Human Transparency describes two models of transparency focusing on the communication of the agent’s awareness of the state of human teammates [19]. The teamwork model focuses on the division of labor between the agent and the human and the human state model focuses on the agent’s communicating their understanding of the human’s cognitive, emotional, and physical state [19]. This delineation of models is particularly important to keep in mind, given the parallels between human teams and human-agent teams. Robot-to-Human agent transparency models describe information relevant to task performance, while Robot-of-Human transparency models focus on the members of the human-agent team.

2.3 Effects of Transparency

The implementation of features that support agent transparency can have a positive influence on the relationship between the human and the agent working together. Depending on the kind of transparency explored and the amount of information presented to the user, information supporting transparency can influence operator trust, situation awareness, and workload.

Trust.

In the context of a human-agent team, trust can be defined as “the attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability” [16]. In a transparent system, human observation of the agent’s actions—such as its history of action, capability, and reliability—should be supplemented by information concerning the goals, reasoning, and situational information that led to these actions [5, 16, 22]. By providing this information, which supports transparency, the human’s trust in the agent can be reasonably informed and more solidly held [7, 16]. However, trust is continuously updated [7]. If the agent communicates uncertainty or reports incorrect information, the human operator can use that information to calibrate their trust, matching their trust with the system’s capabilities, leading to appropriate use [7, 16].

With the incorporation of agents into teams as equal team members, however, humans and agent system designers must account for both trust in the agent as an automated system as well as trust in a teammate to act in the best interests of the team [9, 16]. In pursuit of transparency, system designers attempt to make sure that the agent communicates knowledge about itself, its goals, its underlying reasoning, and its awareness of environmental factors, but this communication of information is most frequently one-way [2, 17, 18].

Situation Awareness.

When humans work with robots, the human needs to maintain situation awareness (SA) in order to make appropriate decisions [23]. Situation awareness refers to the process by which a human attempts to understand their environment and use that understanding to perform competently in a situation as it occurs [24]. When working with agents, SA may include awareness of the environment in which agents may be located or what the agent is doing [23, 25]. Agent transparency can be used to keep the human operator from being pulled ‘out of the loop’ by allowing human operators to focus only on information relevant to the mission [2, 25]. Contextual awareness of an agent is a key factor in the success of human operators, and the communication of the agent’s intent can facilitate overall SA [25–27]. Global SA requires awareness of not only the immediate working environment, but the relationship between the agent and the human within that environment, so transparency specific to the human-agent relationship can support that awareness.

Workload.

A perennial concern is the impact of adding more information to an interface. Supporting transparency could require additional information, which may affect the human’s workload [18]. Although a unitary definition of workload does not exist, it can be conceptualized as the perceived impact of task demand imposed on the human operator and the associated physiological responses [28, 29]. Additional information supporting transparency may cognitively overload the operator, leading to performance decrements [30]. However, if additional information mitigates workload that would have been caused by having to recall or estimate information, the added information may lead to similar, if not lower, levels of workload overall [18, 30]. If overload is a concern, then features that reduce workload to a more manageable level are preferred. In two studies on joint human-agent decision-making, increased amounts of information supporting transparency were added to an interface without a significant increase to workload [31, 32]. Thus, transparency can possibly lead to benefits without a noticeable increase to the human teammates’ workload.

2.4 Communication and the Teamwork Model of Transparency

In the pursuit of transparency, agent interfaces have been designed to communicate the agent’s behaviors, goals, reasoning, and environmental constraints to the human operator in order to facilitate shared intent and shared awareness [17, 18]. While information encompassed by the Robot-to-Human transparency models has been explored as a means of supporting humans’ understanding of their agent teammates, less work has been done exploring the influence of Robot-of-Human transparency. While research concerning automated systems responding to human physiological states and the dynamic division of labor between humans and agents exist, fewer studies focus on the transparency of these systems [2, 19].

A particular area of interest is how agents can express the human’s and agent’s fulfillment of their responsibilities, and how the communication of this information could influence the human’s relationship with their agent team member. In human teams, team members can engage in mutual performance monitoring, a behavior where team members keep track of all team members’ performance, which can be coupled with feedback [33, 34]. This kind of feedback reflects the agent’s model of the human operator back to that selfsame operator, allowing the human and agent to establish a greater shared awareness of both the human’s and agent’s roles in the team and how they fulfill those roles [16, 34].

One study found that humans who worked with a highly autonomous robot attributed more blame to the robot than those who worked with a low autonomy robot [35]. While increased transparency led to a marginally significant reduction in the attribution of credit to other group members working with the highly autonomous robot, the robot’s feedback did not influence credit or blame to the robot or the self [35]. While an explanation of the robot’s reasoning did not influence the aforementioned factors, feedback concerning the robot’s and human’s performance of their roles may do so. While human and robot role responsibility feedback may lead to different credit and blame attribution, feedback concerning human operators’ shortfalls may influence the human’s perception of their own trustworthiness. Similarly to how humans can determine the trustworthiness of an agent through its history of action, capability, and reliability, they may also determine their own trustworthiness by seeing this same information about themselves [5, 16]. However, if the human feels that their own trustworthiness is low, they may feel the need to delegate tasks to the agent [16]. The impact of this kind of information, especially with robots and other embodied agents, is underexplored, despite the implications it may have towards the relationship between humans and agents. The means by which this information is communicated, however, may emphasize benefits or mitigate the possible negative repercussions of this communication. Hence, an exploration of different modes of communication is of interest.

2.5 Multimodal Feedback

Research in the field of agent transparency focuses on the communication of different kinds of information in order to maintain shared intent and awareness [19]. While the content of this communication is important, the means by which it is communicated is notable as well. Content is most frequently communicated through auditory or visual methods [36]. Humans can only process a limited amount of visual or auditory information, and the available mental effort used for each is distinct, so communicating information across multiple modes can extend mental limitations by splitting the burden across multiple channels [37]. Multimodal communication, communication across more than one sensory channel, allows for an increased complexity in communication through the use of redundant or non-redundant signals. The transmission of redundant signals can ensure that the message is received; the transmission of non-redundant signals can communicate two separate messages simultaneously, the modulation of a message, or the communication of an entirely new message [38]. Multimodal communication, can influence workload, which in turn can affect error rate and operator effectiveness [36, 38]. There are several methods in which agents can effectively integrate visual and auditory feedback.

Visual feedback.

The most common form of feedback is visual, with multimodal research investigating the effects of supplementing the visual modality [36]. In learning environments, information is usually presented to learners visually, either through text or graphics [39]. Text feedback has the advantage of facilitating understanding of complex, semantically-rich content [40]. In addition to the content of the message, the social presentation can influence humans’ responses to what was written. Increased etiquette, expressed by automated systems warning operators before giving them support and avoiding interruptions during requested actions, has been shown to lead to better performance and improved trust, though it has hampered situation awareness [7, 41]. The ubiquity of text means that it is frequently supplemented by other methods of communication [42]. Text-based feedback has been paired with graphics and speech, yielding more comprehensible output and more creative solutions [43, 44]. While agents can use disembodied text to communicate information, an agent with a human-like avatar can potentially provide an emotional connection that can create a more positive relationship between the agent and their human operator [42].

Software agents that are depicted using virtual characters are capable of communicating using human-like verbal and nonverbal signals [45]. A virtual character can provide feedback through the use of facial expressions or gaze [7, 45]. Positive facial expressions on pedagogical agents’ characters have been used to facilitate learning, motivate learners, and aid in attitudinal learning [45]. Gestures as social cues can be used to draw attention to an important feature in an interface or can be coupled with speech to improve the recall of verbal information and captivate the human operator [42, 46]. When robot gestures were combined with synthetic speech, it was more positively evaluated than when it communicated using speech alone [47].

The desired type of agent feedback—gestures, facial expressions, and so on—informs the design of a robot. A robot’s shape can potentially influence how it is seen; in one study, spider-legged robots were seen as more aggressive than wheeled robots, while robots with arms were seen as more intelligent than those without [48]. An agent with a human face or avatar may be more likely to engender human-like treatment from their human teammates, but human-like treatment may not be the desired response [49]. These different shapes are conducive to different gestures. A robot with arms and legs can make different gestures than a robot with no arms and wheels. The addition of expressive lights can add an entirely new dimension in communication, with speed, regularity, and color providing an avenue through which messages can be communicated [50]. The major limit in visual communication seems to lie in physical feasibility and human understanding.

Auditory feedback.

Sound has often been used as a social cue to focus people’s attention and provide feedback [36]. One study found that when earcons (i.e., abstract musical tones) were used to indicate movement of an interface element, it led to slower completion of a highlighting task than visual highlighting alone [36]. When abstract tones were used as robot feedback, participants did not extend assistance to it as much as when it had a voice, either synthetic or human [51]. When working with a mix of synthetic and human speech, users’ perception of their own performance was higher than users who only received a synthetic voice, but their actual performance in a series of office tasks was worse than their counterparts who only received synthetic speech [52]. Synthetic speech, unfortunately, is judged based on its intelligibility, naturalness, and acceptability to the human [53]. Participants exhibited faster response latencies when listening to natural voice compared to those who were listening to a synthetic voice [54]. Overall, speech is suited for presenting short, semantically simple content which carries only essential information [40, 42].

3 Experimental Approach

The exploration of human-agent teams, agent transparency, and multimodal communication has set the stage for a proposed approach to the investigation of transparency in human-agent teams. Specifically, this investigation seeks to make the case that the influence of an agent’s multimodal feedback in response to human operators’ meeting and not meeting their responsibilities within the division of labor is an area of research that has not yet been plumbed.

3.1 Experimental Considerations

A central question in the exploration of information concerning division of labor is how that information is expressed to the human operator. In tasks where humans and agents must work together to complete a task, the teamwork model of transparency suggests that the agent inform the human about their respective responsibilities and if they are being fulfilled. If an agent communicates to its human counterpart that they are fulfilling or failing to fulfill their responsibilities, how will that communication influence the human’s performance and their relationship with that agent? Establishing how feedback concerning division of labor can influence the relationship between human and agent may be useful. Additionally, a human-appearing agent leads to human-like expectations from it, which may or may not be desired [17, 49]. Will human-like social cues and non-human-like social cues yield equal benefits of transparency? Would teamwork transparency alone provide the same benefits that robot-to-human transparency does, or would they work better together? Scales pertaining to Workload, Trust, and Situation Awareness are useful indicators to determine the extent to which the information supporting teamwork transparency facilitates a beneficial relationship between the agent and the human.

3.2 Research Questions

Information pertaining to division of labor can be communicated multimodally, with human-like interfaces, using a human-like avatar and natural voice feedback, or non-human interfaces, using flashing lights and beeps. One aim of this approach is to determine the extent to which a human-like presentation of information supporting transparency influences operator behaviors. Human-like robots have resulted in a specific behavioral pattern from human operators [49].

Additionally, information supporting the teammate transparency model may require precise communication, so the use of non-redundant multimodal messaging should be explored as well. Non-redundant multimodal signals can be used to communicate modulated messages, which allows for a finer-tuned message concerning division of labor [38].

Furthermore, another area that bears exploring is the coordination of different models of transparency. Would a model of transparency, such as the SAT model, benefit from the addition of information supporting teamwork transparency? Would the operator attend to features of the interface supporting their situation awareness if periodically reminded of their responsibilities? Given the parallels between human teamwork and human-agent teamwork, determining the utility of information supporting teamwork transparency on its own and combined with a task-oriented transparency is a key area of exploration.

4 Evaluation

Dependent variables pertaining to the utility of human-agent transparency include performance, how well the agent supports the operator, and the human operator’s relationship with the agent. Performance, a key dependent variable, can include the successful completion of decision-making tasks, comprehension of presented information, and search tasks. The human’s performance and perceptions of their performance should be measured. An agent can support the operator by assisting in the maintenance of situation awareness and workload in a way that serves performance, so these two factors should be evaluated. The human operator’s relationship with the agent includes trust, which influences automation use, disuse, and overreliance [16]. Given the large impact that trust can have, it should be evaluated as well. Additionally, it is also important to evaluate the human’s perception of the agent, including technology acceptance and perceived usability.

5 Conclusion

To maintain the benefits of keeping the human operator ‘in the loop,’ autonomous agents must maintain transparency. As agents are increasingly tasked to act as teammates, however, they need to also communicate information to support teamwork, rather than just information to support operators’ tasks. Like mutual performance monitoring can aid human teams, the presentation of information supporting the teamwork model of transparency should benefit human-agent teams. The communication of role responsibility informs operators of the agent’s understanding of the humans’ and agents’ responsibilities in the system and how those responsibilities are being fulfilled. Providing feedback about the human’s actions as a team member may allow for greater teamwork between humans and agents. Presentation of this information by an agent may have unintended consequences, though, so research should look at presenting information using both visual and auditory communication methods, both human-like and non, and with other forms of automation transparency. This research will inform the design of agents for human-agent teams where an authentic artificial teammate is desired. As agents become more complex and are able to do more, our understanding of how humans treat their teammates, human or not, becomes even more necessary to facilitate effective performance from human-agent teams.