Keywords

1 Introduction

The pilot SA was a vital index to evaluate the design of cockpit display interface. It was indicated that the pilot SA had a direct correlation to flight safety in relevant research. A correct judgment and decision could be made more rapidly and efficiently by pilot to achieve a higher level of flight safety when facing to higher SA [1, 2]. The statistical result of aviation accidents revealed that 35.1% non-major accidents and 51.6% major accidents were caused by the failure of pilot’s decision-making, and the main reason was the lack of SA or SA error instead of the error in decision-making [3]. It was a common conception of SA in aviation ergonomics, but there still existed no strict definition. The three level definition presented by Endsley was a classic one and commonly accepted by other scholars [4].

It was considered that the operational definition of SA was mainly to measure the result of procedure (such as whether the event was comprehended by operator or not) instead of the process to the acquisition of relevant SA. For example, the pattern for pilot to aware the dangerous terrain was not an important issue, however, the measurement of SA was only to evaluate whether it had been conscious of or not. So, the genuine measurement of SA should be concentrated on the dynamic element. The common measurements were situation awareness rating technique (SART), the measurement based on memory retention, the measurement based on operational performance, and the measurement based on physiological indices [5]. A combination of measurement based on memory retention, subjective assessment, and interview of critical event were employed by Paul to measure SA in experiment task [6]. The SAGAT method was used to measure SA by Riley, and meanwhile the immersion and mental workload was also investigated by questionnaire to analyze how the measurement of SA and attention allocation impacting the explanation of immersion cognition [7].

The conception of SA had already drawn extensive attention in all walks of life. The phenomenon not only related to the design of display supporting SA, but also to the happening reasons of disaster and accident. It should be ensured that the information which ought to be monitored in the design of interface should be presented to the pilots in a clear and comprehensible way under current automation. A tiny variation in the presenting format of information display would affect pilot SA. Therefore, the scheme of interface design should pay more attention to the factors with potential possibilities, and a full-scale experimental measurement would be made in multiple tasks [8]. The afterward measurement was the main method in current research, but the forecasting method was yet rarely that the prediction method based on the three levels of SA was even deficient. Moreover, the prediction and comprehend of pilot SA should be made to evaluate whether the interface design was good or bad in practical aviation industry, and this would be a scientific basis and theoretical foundation for further optimization of interface design and the reduction of human error.

2 Method

2.1 Subject

Nine graduated students from Beihang University were recruited as subjects in the experiment, with normal or corrected to normal eyesight, non-color blind and basic knowledge of civil aviation. They were all informed with the detail of experiment task and procedure, and voluntarily agreed to participate in the experiment.

2.2 Apparatus

The experiment platform was selected as simulation cockpit for flight simulation task, as shown in Fig. 1. The hardware of experiment platform was composed of main server, LED monitor, steering wheel, engine throttle, automatic control panel, seat and cockpit shell. The software of experiment platform was composed of flight simulation formula, which drove multi-screen to provide proximate actual experience of flight operation through simulation control devices.

Fig. 1.
figure 1

The experiment scenario

Full size image

2.3 Experimental Design

Boeing 737–800 was chosen as experiment airplane, and weather was set as sunny day in summer. The flight task was selected as right-hand traffic pattern flight based on the current airport runway, which was initialized in Snapshot with magnetic heading 179, airspeed 220, auto-pilot heading hold 179, altitude hold 2800, and flap 5. The specific task operation was followed with the standard operation procedure according to flight manual of Boeing airplane, and was adjusted to the current situation of flight simulation platform.

The subjects were demanded to take several adaptive exercises of flight task till they were capable to complete the entire experiment task independently. SAGAT method was used to measure SA level in memory probing fashion, which required the subjects to make immediate response to the shown question as the flight task was randomly frozen. According to the purpose of current study, the experiment task was divided into take-off and climbing, cruising flight, as well as approach and landing. The relevant operation process was shown in Table 1.

Table 1. Task operations of traffic pattern flight
Full size table

2.4 SAGAT Measurement

SAGAT method was commonly used evaluation based on memory probing technology, which was applied to lead task process to randomly pause and substitute the display interface into questions, basically in choice form, that required the subjects to make immediate response according to the current status of task scenario. The specific question designed in the experiment was included with twenty-three task-related questions triggered by different conditions.

3 Results

Each question of SAGAT method in the experiment was referred to each key step of flight operations, involving with flight information on primary flight display, navigation display, and mode control panel. However, the content of SAGAT questions could be repeated, yet their triggering conditions were varied. And the experiment results were analyzed respectively by experiment task operations, and the tendency of accuracy and response time of SAGAT measurement was shown in Fig. 2.

Fig. 2.
figure 2

The experimental results of accuracy and response time (ms) of SAGAT

Full size image

In addition, the SA measurement was investigated separately according to each flight task phases mentioned in Table 1. For take-off and climbing phase, the dynamic results of SAGAT measurement were shown in Table 2.

Table 2. Results of SAGAT measurement during take-off and climbing phase
Full size table

Based on experiment results of accuracy and response time, the experiment value of SA was determined by accuracy and modified in consideration of response time. The specific quantification was shown below.

$$ {\text{SA}}_{i} = {\text{SAGAT accuracy}}_{i} \times \frac{{{\text{SAGAT response time}}_{i} }}{\text{SAGAT average response time}} $$
(1)

And the experiment value of SA was calculated as 0.74, 1.30, 1.15, 0.75 and 0.72 according to Eq. 1. However, the situation might occur that such SA could be over 1.0, where the absolute value of SA would not be discussed because the purpose of this paper concentrated on the dynamic measurement of SA and the validation of prediction model. Therefore, the experiment value was proportionally normalized into 0 to 1.0, and that of take-off and climbing phase was shown in Fig. 3.

Fig. 3.
figure 3

Dynamic tendency of SA results during take-off and climbing phase

Full size image

For cruising flight phase, the dynamic results of SAGAT measurement were shown in Table 3, and the modified experiment value of SA was shown in Fig. 4.

Table 3. Results of SAGAT measurement during cruising flight phase
Full size table
Fig. 4.
figure 4

Dynamic tendency of SA results during cruising flight phase

Full size image

For approach and landing phase, the dynamic results of SAGAT measurement was shown in Table 4, and the modified experiment value of SA was shown in Fig. 5.

Table 4. Results of SAGAT measurement during approach and landing phase
Full size table
Fig. 5.
figure 5

Dynamic tendency of SA results during approach and landing phase

Full size image

4 Validation of Prediction Model

In consideration of memory retain and attention allocation [9,10,11], the dynamic prediction model of SA was established by furthering Wu’s model [12], which illustrated the average level of SA influenced by each situational element at time T.

$$ SA\left( {t_{i} } \right) = \mathop \sum \limits_{i = 1}^{n} \left( {1 - 0.5k_{i} } \right)u_{i} A_{i} $$
(22)

Where n was the number of situational element, ui was the fuzzy membership of information priority, Ai was the attention allocated on situational element i, and Ki was individual difference concerned with the capability of information perception achieving to understanding.

According to experimental task design, the cockpit display interface was divided into nine situational elements: 1. airspeed, 2. altitude, 3. attitude, 4. heading, 5. navigation, 6. landing gear, 7. flap, 8. engine, 9. auto-pilot system. Taking “engage auto-pilot” (the last operation mentioned in Table 2) as example, the quantification of SA was shown below. For information priority, the situational element of No. 9 auto-pilot system was the highest, and priority ordering was followed with relevance between each situational element and current task operation: 9>3>5>2, 4>1>6, 7>8. For cognitive activation, the situational element of No. 9 auto-pilot system was also the highest as 1.0, and others was set as 0.5. And for memory retain, each situational element was declined with coefficient 0.9 or 0.8 based on its informativeness. The specific value of SA quantification was shown in Table 5, and the current SA was calculated as 0.473.

Table 5. Calculation of SA in each situational element
Full size table

The dynamic prediction model was validated in regression analysis of experimental and predictable SA, respectively in different phases of take-off, cruising flight and landing. Both accuracy and response time of SAGAT measurement were concerned as experimental result of SA, and human error was also taken into account since it might be caused by misunderstanding of situational element. Therefore, the regression analysis of take-off and climbing phase showed that the prediction model achieved reasonable agreement with the experimental result with R2 = 0.863, as shown in Fig. 6.

Fig. 6.
figure 6

Regression analysis of SA during take-off and climbing phase

Full size image

And the regression analysis of cruising flight phase showed that the prediction model achieved certain agreement with the experimental result with R2 = 0.548, as shown in Fig. 7. However, this experimental result was calculated without human error, it was modified by the occurring times of human error according to each task operations so that the experimental SA declined from 0.94, 0.88 to 0.88, and 0.70. Therefore, the regression analysis of the modified results showed better validation with R2 = 0.755, as shown in Fig. 8.

Fig. 7.
figure 7

Regression analysis of SA during cruising flight phase

Full size image
Fig. 8.
figure 8

Regression analysis of modified SA during cruising flight phase

Full size image

Moreover, the regression analysis of approach and landing phase showed that the prediction model achieved considerable agreement with the experimental result with R2 = 0.835, as shown in Fig. 9.

Fig. 9.
figure 9

Regression analysis of SA during approach and landing phase

Full size image

5 Cognition Capability Testing

The influencing factors of SA included with both internal and external factors, and the internal ones were mainly related to the characteristics of perception and cognition capability as well as expertise skills. Since the subjects recruited in this experiment were equally trained and their experience of flight simulation task was almost the same, the influence of perception and cognition capability was primarily examined and analyzed, which was involved with basic response time, spatial rotation, short-term memory, and attention inhibition. The basic response time was tested in E-prime program according to classic paradigm of subtract method. And the others were tested online based on task of Rotation, Digit Span, and Double Trouble on the website of Cambridge Brain Sciences [13]. The subjects were required to take such tests accordingly after one entire exercise.

Due to the small sample of nine subjects, the results of cognition capability were partially accorded with normal distribution except for short-term memory and simple reaction time, as shown in Table 6.

Table 6. Results of cognition capability testing
Full size table

Correlation analysis was used to examine the relationship between cognition capability and experimental results of SA, which revealed significant correlation between response time of SAGAT measurement and simple reaction time (r = 0.795, p = 0.010), accuracy of SAGAT measurement and discriminative reaction time (r = 0.702, p = 0.035) as well as spatial rotation (r = 0.704, p = 0.034). Therefore, individual difference could be found between the subjects so that their SA varied in correlation with cognition capability.

6 Conclusion

In conclusion, the dynamic SA measured by SAGAT method showed good agreement with prediction model based on flight simulation task of right-hand traffic pattern flight. Moreover, the characteristics of cognition capability revealed significant correlation with SA.