Keywords

1 Introduction

Distributed simulations can improve fidelity and reduce costs by taking advantage of facilities and personnel in different locations [1]. For instance, they can allow data collection over multiple weeks, which may not be feasible if personnel need to relocate for data collection. This paper describes the use of distributed simulation to investigate Human-Autonomy Teaming (HAT). HAT is of increasing interest because of recent advancements in the speed and quality of automation technology, advances in speech recognition, and more collaborative behavior [2]. We will describe teaming with an autonomous system that was initially based on the Emergency Landing Planner (ELP), a decision support tool that generates recommendations for emergency landing sites. Three simulations will be described, showing a progression from integrating automation into a flight following ground station, increased HAT functionality for the ground station, and migration of the autonomy and HAT functionality into the flight deck.

1.1 The Emergency Landing Planner (ELP)

The ELP is a decision support tool developed to assist pilots in choosing the best emergency landing site for disabled aircraft [3, 4]. The ELP was initially designed for flight deck use in transport aircraft. The ELP was accessed through the Flight Management System (FMS), Cockpit Display Unit (CDU), specifically by a prompt on the CDU Departure/Arrival page. Landing recommendations were displayed on the CDU on a new page that permitted examination and selection of each alternative.

Four categories of risk factors served as inputs for computations: (1) en route (2) approach (3) runway and (4) airport. The primary factors considered for en route risk were controllability of the aircraft, distance and time to the site, complexity of the flight path, and weather along the route. Approach risk took into consideration weather along the approach path, characteristics of the instrument approach, ceiling and visibility at the airport, and population along the approach path. Runway risk included factors such as length and width of the runway, landing speed, and relative wind speed and direction. Finally, airport risk incorporated the availability of emergency facilities at and near the site. The routes generated by the ELP specified paths from the aircraft’s current position to a destination runway. The following obstacles were considered in the route computations: (1) terrain (2) hazardous weather and (3) special use airspace.

Due to limited screen real estate on the CDU, the ELP could only provide brief explanations for the reasoning leading to the recommendations. Explanations were limited to two-character abbreviations showing the main risks associated with each option. For example, the code “CE” showed that the cloud ceiling was close to minimums for the approach.

2 Simulation 1: Dispatch Autonomy

The ELP was integrated into a prototype ground station to help resolve scripted off-nominal or high workload events by selecting a divert airport [5, 6]. The integration allowed ground station simulations to be run in one building while the ELP was hosted in its development lab in another building.

In addition to the ELP, the ground station replicated many flight deck displays, including the Mode Control Panel (MCP), Control Display Unit (CDU), and Navigation Display (ND) (Fig. 1a). The output from the ELP was displayed with flight tracking tools and the ELP on a Traffic Situation Display (TSD) to the right (Fig. 1b). Finally, tools that helped remote crews keep track of their roles (pilot flying vs pilot not flying) and responsibilities with respects to who was handling what controls (i.e., speed, heading, altitude, or CDU) were presented to pilots on displays in area “c” of Fig. 1. The main tool of interest in this paper is the ELP. For a more complete description of the tools in Fig. 1a, see Brandt et al. [5]. For details regarding the collaboration tools in Fig. 1c, see Ligda et al. [7].

Fig. 1.
figure 1

Dispatch ground station setup: (a) replicated flight deck displays for the chosen aircraft; (b) flight tracking displays with ELP recommendations; and (c) crew collaboration tools including sharable charts.

Full size image

The ground station version of ELP was invoked by the ground operator with a button located on the lower left corner of the TSD in area “b” of Fig. 1. The ELP then generated a rank-ordered list of options to discuss with the pilot as shown in Fig. 2 (“EAPs” refers to the Emergency Action Plans represented by the options). An airport may appear more than once in the list, with different runways having different risks. Once the list of landing options was displayed, the ground operator could then select an option. The proposed route for the selected option was displayed on the TSD as a route drawn from the nose of the aircraft symbol to the recommended airport. The current route was shown along with the proposed route. The proposed route could then be sent to the pilot for consideration. When the flight deck executes a recommendation, it becomes the active route and changes to magenta on the TSD.

Fig. 2.
figure 2

The Emergency Landing Planner (ELP) recommendations shown on the Traffic Situation Display (TSD).

Full size image

Post-trial questionnaires were used to evaluate the recommender system across four different criteria: (1) trust; (2) transparency; (3) effectiveness; and (4) satisfaction. Results from the post-trial questionnaires showed that participants did not clearly reveal any trust or see transparency in the recommender system. However, pilots did appear to find the recommender system to be effective in supporting them with high workload or off nominal situations, and interactions with the system appear to have been satisfactory. Pilots also reported in post simulation surveys a desire to have better explanations for those recommendations (refer to [5, 6] for more details).

3 Simulation 2: Ground HAT

Given the feedback about the lack of ELP transparency from the previous study, a Human-Autonomy Teaming approach was used to design new interactions with the ground station [2, 8, 9]. HAT features were derived from three tenets:

  • Operator Directed Interface: An allocation of tasks based on operator direction and context allows for a more flexible system and keeps the operator in the loop.

  • Transparency: Providing the system’s rationale for selecting a particular action helps the human understand what the automation is doing and why.

  • Bi-Directional Communication: For automation to act as a teammate, there needs to be two-way communication about mission goals and rationale.

Enhancements were made to the ELP to allow interactions using these HAT features, and this version is referred to as the Autonomous Constrained Flight Planner (ACFP) to reflect its new capabilities.

The components of the next iteration of the ground station included the Aircraft Control List (ACL), the Traffic Situation Display (TSD), and additional displays (Fig. 3). The ACL was the primary tool for managing multiple aircraft and switching the focus between aircraft (see Fig. 3A). It displayed a timeline, alerting information, and HAT features.

Fig. 3.
figure 3

Ground station HAT setup: (A) Aircraft Control List (ACL), augmented with timeline, alerting information and HAT features. (B) Traffic Situation Display (TSD). (C) Flight controls and displays for the selected aircraft in read-only mode. (D) CONUS map and charts.

Full size image

An Operator Directed Interface was implemented using the playbook approach to set goals and manage roles and responsibilities between the operator and the automation [10]. It provided 13 different plays the operator could call to address simulation events. When the operator selects a play, the ACFP is initiated with preset weights, and the corresponding play checklist appears on the display identifying operator tasks in white and automation tasks in blue (see Fig. 4).

Fig. 4.
figure 4

Operator Directed Interface for calling plays in the HAT condition and associated checklist of roles and responsibilities. (Color figure online)

Full size image

In addition to risk factors, the ACFP took into account factors such as fuel, distance, and services available. It also had the capability of weighting these factors differently based on the situation. Transparency was implemented by explicitly showing the factors and weights of the recommended divert airports (see Fig. 5). Additionally, the scores for the ACFP factors were translated to meaningful values (e.g., presenting nautical miles (nm) instead of a score). Figure 5 shows the result of a Medical Emergency play being called, which resulted in the distance to medical facilities (Medical row) and time to destination (ETA row) given more weight than other factors. As a result, Cheyenne (KCYS) was the top recommendation showing a trauma care facility 3 nm from the airport. Although Denver (KDEN) was closer, the trauma care facility is further from the airport.

Fig. 5.
figure 5

Transparency and Bi-Directional Communication in the HAT condition implemented by ACFP factors (on bottom) and weights (on top).

Full size image

To implement Bi-Directional Communication, weights were preset for each play and presented in slider bars (top of Fig. 5). However, the operator was able to negotiate with the system by altering these weights to what the operator considered appropriate for the situation. The operator can change the weights and see how the divert recommendation is affected. Using the example shown in Fig. 5, if the operator decided that distance to the airport was a higher priority than available medical facilities, the operator could adjust the ACFP weights and find new recommendations.

In addition to presenting information graphically and accepting mouse input, voice recognition and voice synthesis technologies supported the ability to perform some functions by voice. Specifically, operators could select specific aircraft, invoke the rerouting tool, and request briefings (brief summaries of the state of the system, an aircraft, or an airport), using voice commands. Briefings were given in both vocal and textual format. Alerts and certain system changes (e.g., aircraft landing) were also announced vocally.

As in the last simulation, ground station simulations were run in one building while the ELP was hosted in its development lab in another building. Overall, dispatchers and pilots preferred the ground station with HAT features enabled over the station without the HAT features. Participants reported that the HAT displays and automation were preferred for keeping up with operationally important issues. Workload in the HAT condition was found to be lower, as measured by both subjective rating and eye-gaze duration. Participants agreed that the automation and displays did a good job of integrating information, and they liked the ACFP (for example, ‘‘The ACFP is outstanding… We like to be able to verify stuff, so what is really cool is you guys have that ability, you don’t just blindly trust, you can literally look at the ATIS and say, ‘Ah! I think that is pretty accurate.’’’). For more details of the simulation and results please see [8, 9, 11].

4 Simulation 3: Flight Deck HAT

In order to test the generality of the effectiveness of HAT features, the decision support functionality of the ground station was migrated to a tablet in a flight deck [12,13,14]. A multi-site distributed simulation allowed the use of the flight deck simulator in Cedar Rapids Iowa while the ACFS autonomy and sim management was hosted 1500 miles away in Long Beach California. The simulation was monitored by NASA in Mountain View California (see Fig. 6).

Fig. 6.
figure 6

Locations of distributed simulation components

Full size image

As in the ground station, the ACFP provided divert options in off-nominal simulation scenarios. Plays were again used to provide an Operator Directed Interface, but the human-automation checklist was modified to enable cockpit-centric actions. Transparency of ACFP decisions was demonstrated by displaying the factors and weights of recommended divert airports. And Bi-Directional Communication between pilot and ACFP was implemented with factor weight sliders. As with the ground station, voice recognition and voice synthesis technologies allowed the pilot to have voice interactions with the ACFP.

Data were collected from 12 pilot subjects over a six week period. Distributed simulation allowed researchers to remain in their respective cities. As with the ground station, positive evaluations were given for HAT functionality [12,13,14]. A significant preference for HAT over No HAT was found for workload, efficiency, information integration, and keeping up with important tasks. There was significant agreement on “I would like to have a tool like the ACFP with HAT features to use with real flights”.

Pilots also gave positive feedback:

  • “HAT made a 75% reduction in workload…it really proved its worth to me.”

  • “HAT was definitely helpful. A great resource for high workload operations.”

  • “Great experience. You guys are doing great work here.”

Please see Battiste et al. [12], Cover et al. [13], and Strybel et al. [14] for more details of this simulation and its results (Fig. 7).

Fig. 7.
figure 7

Flight deck HAT setup: (A) tablet for interacting with HAT features.

Full size image

5 Conclusions

This paper describes the use of distributed simulation to investigate Human-Autonomy Teaming with three simulations, showing an iterative design process, progressing from integrating automation into a flight following ground station, to increased HAT functionality for the ground station, and to migration of the autonomy and HAT functionality into the flight deck. The distributed nature of the simulations provided high efficiency by allowing researchers to work in their own labs over an extended period of time for simulation set-up and data collection. It also provided high fidelity by enabling access to a flight deck environment. Also, it managed costs by taking advantage of facilities and personnel in different locations.

Using these simulations, Human-Autonomy Teaming (HAT) factors were found to enhance ground station and flight deck environments. Conditions with HAT features were preferred by simulation participants to conditions without. Subjective and objective measures of workload were lower with HAT features.

These results indicate that future investigations into aircraft support would benefit from the addition of HAT factors and the use of distributed simulation.