Keywords

1 Introduction

Aviation accidents have been contributed mostly by human factors. Pilots are the most active elements in flight activities, so pilots’ operation performance could affect flight safety directly. Statistics by the global aviation accidents, the pilot is the key factor of the flight safety. Many results show that more than 60% of the flying accidents were caused by pilot errors, it is the main contributing factor of aviation accidents and incidents [1,2,3]. According to the statistics of recent years on commercial flight accidents in China, the percentage of accidents caused by the crew factors is rising. The flight crew factors contributed to 61.54% of accidents in 2001–2006 [4], and it increased to 64.58% in 2006–2015 [5]. In fact, no matter what causes the flight accident, it would eventually behave in the operation behavior [6, 7]. Therefore, it is of great practical significance to improve the pilot’s operation performance and reduce the crew errors to prevent the aviation safety accident.

The flight quick access recorder (QAR) is a system that can acquire aircraft operational data easily. It includes airborne equipment for recording parameters such as speed, attitudes, altitude, control deflections, etc. A ground software station for developing software to process the QAR data and obtain the required meteorological quantities by taking into account the aerodynamic factors of the aircraft types commonly operated by the local airlines [8]. The QAR can record all kinds of aircraft parameters, pilot operation parameters, environmental features, and alarm information during an entire flight. The practice has proved that QAR data are helpful for improving flight safety management and quality control [9]. However, the data have been rarely utilized in research.

This paper described the main features of the QAR data analysis system and illustrated its application in evaluating pilot performance. The main purpose of this study is to develop a flight operation performance studies. This evaluation system was based on flight quick access recorder data and a risk evaluation model. The system is expected to be used to store, analyze, evaluate, pre-alarm and improve the performance of the flying operation of the pilot, to provide practical technical support for airlines to monitor pilots’ flight operation performance and flight risk.

2 Methodology

2.1 Quick Access Recorder Data

QAR data includes a large number of flight, operation, environmental and other types of airplane information, it mainly used in aircraft fault detection and simple operation management in current. The use of a large number of data is lack of system, and is not effective, resulting in the waste of information.

The flight QAR data, which is based on related operational rules and regulations, is used by commercial airlines to monitor and analyze the aircraft status and pilot operation performance in flight. When flight data exceeds the prescriptive normal range [8], a QAR Exceedance Event [10] or Unsafe Event is recorded by our system. The QAR Exceedance Event was divided into two levels. The first level was called Detect Limit, while the second level was called Alert Limit. Taking the Boeing 737-800 model as an example, the model had 108 types of QAR Exceedance Events, and a part of the types were shown in Fig. 1.

Fig. 1.
figure 1

QAR Exceedance Event standard sample

Full size image

Exceedance Events may not lead to serious consequences. However, they could increase the probability, and could bring potential risks to aircraft and even passengers.

2.2 Evaluation Model

At present, the use of QAR data by most airlines was only limited to QAR data analysts to extract the data after flight execution for the traditional management of QAR Exceedance Events. That is, according to the level and frequency of QAR Exceedance Events standards to monitor flight and evaluate the pilots performance. Most airlines give up research on the data and ignore the value of the data. In a sense, the flight risk is ignored. A large amount of QAR data can reflect the pilot’s operational characteristics at all stages of flight clearly. Taking QAR data of a flight fleet in a long period of time as the sample space, we studied the probability distribution of the entire fleet \( P_{fleet} \) according to statistical principles. With the same methodology, the probability distribution of one single pilot was calculated, and then compared with \( P_{fleet} \), in order to evaluate and predict the pilot’s risk of Exceedance Event.

Quantitative evaluation method is one of the important methods for risk assessment. Generally, statistical and computational methods were used to multiply the probability of risk occurrence and the severity of its consequences to obtain the risk value. This method has less qualitative analysis, and it has higher accuracy [11, 12]. Based on the large sample statistics of QAR data, it was found that most flight performance parameters, such as touchdown distance, vertical acceleration, and pitch angle, are approximately normal distribution in large sample space (n > 100) [13]. Therefore, we can set a healthy fleet in a stable environment, each kind of flight parameter distribution will be approximately a normal distribution in a long period. Then the occurrence probability of the various parameters of the aircraft fleet will also tend to be relatively stable.

The risk value is obtained by multiplying the probability of the occurrence of the risk with the severity of the consequences. As a result, the severity of each pilot QAR Exceedance Event is actually the same. In evaluating the risk of a pilot Exceedance Event in a certain flight fleet, we only need to calculate the probability of the Exceedance Event occurrence of the pilot, while evaluating the pilot operation performance for a period of time. It is possible to calculate the probability of the Exceedance Event occurrence of each parameter of the pilot in a period and compare it with the stable value of the corresponding Exceedance Event occurrence of the flight fleet. Finally, the pilot’s operation performance was evaluated by evaluating each of the pilot’s Exceedance Event risk levels. Based on the above analysis, taking the Boeing 737–800 model as an example, the evaluation model of pilot operation performance was written as follows:

$$ p_{fleet,i} = \frac{{D_{fleet,i} }}{{{\text{N}}_{fleet} }}\quad \quad \left( {i = 100,101,102 \ldots 207} \right) $$
(1)
$$ p_{fleet} = \frac{{\sum\limits_{i = 100}^{207} {D_{fleet,i} } }}{{{\text{N}}_{fleet} }}\quad \left( {i = 100,101,102 \ldots 207} \right) $$
(2)

In formulas 1 and 2, \( p_{fleet,i} \) is the probability of a Detect Limit event occurrence of the entire fleet. \( p_{fleet} \) is the probability of all kinds of Detect Limit events occurrence of the entire fleet. \( D_{fleet,i} \) is the number of a Detect Limit event of the entire fleet. \( {\text{N}}_{fleet} \) is the number of the flights of the entire fleet. \( i \) is the code of QAR Exceedance Event, from 100 to 207.

$$ p_{pilot,i} = \frac{{D_{pilot,i} }}{{{\text{N}}_{pilot} }}\quad \left( {i = 100,101,102 \ldots 207} \right) $$
(3)
$$ p_{pilot} = \frac{{\sum\limits_{i = 100}^{207} {D_{pilot,i} } }}{{{\text{N}}_{pilot} }}\quad \left( {i = 100,101,102 \ldots 207} \right) $$
(4)

In formulas 3 and 4, \( p_{pilot,i} \) is the probability of a Detect Limit event occurrence of a pilot. \( p_{pilot} \) is the probability of all kinds of Detect Limit events occurrence of a pilot. \( D_{pilot,i} \) is the number of a Detect Limit event of a pilot. \( {\text{N}}_{pilot} \) is the number of the flights of a pilot. \( i \) is the code of QAR Exceedance Event, from 100 to 207.

$$ P_{fleet,i} = \frac{{A_{fleet,i} }}{{{\text{N}}_{fleet} }}\quad \left( {i = 100,101,102 \ldots 207} \right) $$
(5)
$$ P_{fleet} = \frac{{\sum\limits_{i = 100}^{207} {A_{fleet,i} } }}{{{\text{N}}_{fleet} }}\quad \left( {i = 100,101,102 \ldots 207} \right) $$
(6)

In formulas 5 and 6, \( P_{fleet,i} \) is the probability of a Alert Limit event occurrence of the entire fleet. \( P_{fleet} \) is the probability of all kinds of Alert Limit events occurrence of the entire fleet. \( A_{fleet,i} \) is the number of a Alert Limit event of the entire fleet. \( {\text{N}}_{fleet} \) is the number of the flights of the entire fleet. \( i \) is the code of QAR Exceedance Event, from 100 to 207.

$$ P_{pilot,i} = \frac{{A_{pilot,i} }}{{{\text{N}}_{pilot} }}\quad \left( {i = 100,101,102 \ldots 207} \right) $$
(7)
$$ P_{pilot} = \frac{{\sum\limits_{i = 100}^{207} {A_{pilot,i} } }}{{{\text{N}}_{pilot} }}\quad \left( {i = 100,101,102 \ldots 207} \right) $$
(8)

In formulas 7 and 8, \( P_{pilot,i} \) is the probability of a Alert Limit event occurrence of a pilot. \( P_{pilot} \) is the probability of all kinds of Alert Limit events occurrence of a pilot. \( A_{pilot,i} \) is the number of a Alert Limit event of a pilot. \( {\text{N}}_{pilot} \) is the number of the flights of a pilot. \( i \) is the code of QAR Exceedance Event, from 100 to 207.

3 System Design

In the last section, the flight operation performance evaluation model was established based on flight QAR data and quantitative evaluation method. There were 108 evaluation indexes, and we can select one or several of them for combination to evaluate the pilot’s flight operation performance. For example, it’s possible to use three landing operation performance evaluation indexes (touchdown distance, vertical acceleration, and pitch angle) [14, 15] of the pilot to evaluate the pilot’s landing operation performance objectively according to the model and algorithm.

In this section, the flight operation performance evaluation system will be introduced. The flight operation performance evaluation system was designed to include 7 modules: pilot management, QAR event inquiry and statistics, pilot operation performance evaluation, training and upgrade program, early warning, user center and system administration. The hierarchical structure of the system and each sub-function module of the system were shown in Fig. 2.

Fig. 2.
figure 2

Hierarchy diagram of flight operation performance evaluation system

Full size image

4 System Development

4.1 Development Environment and Process

We adopt LNMP architecture, a popular web development technology to develop the database and system. The system is hosted by Linux operation system with Nginx as web server. MySQL is used as database server and we have one slave database hosted by another machine for data backup. We use PHP as our programming language. For the frontend development, we use bootstrap.css for both desktop and mobile friendly visiting.

4.2 System Interface and Functions

The developed Flight Operation Performance Evaluation System (FOPES) includes 7 modules, such as QAR event inquiry and statistics, pilot operation performance evaluation, training and upgrade program, and early warning. The main interface was shown in Fig. 3. The main interface includes a menu bar and links to 6 functional modules.

Fig. 3.
figure 3

Main interface of FOPES

Full size image

The core function of this system is to evaluate the pilot’s operation performance through using flight data and evaluation model. Firstly, clicking the “Pilot Operation Performance Evaluation” icon in the main interface. Next, set the parameters such as the pilot’s name, the evaluation item and the time period. And then, the system will enter the calculation page. When the calculation is completed, the system will provide a prompt box and jump to the evaluation result page, as shown in Fig. 4. The evaluation results of this pilot can be sent to the training department for the targeted training.

Fig. 4.
figure 4

Evaluation results of flight landing operation performance

Full size image

Another important function of FOPES is to provide users with QAR event inquiry and statistics. Users can enter the event inquiry statistics page by clicking QAR event inquiry and statistics on the main interface. After inputting the information regarding the captain and the time period, the system will indicate the relevant statistical results, as shown in Fig. 5.

Fig. 5.
figure 5

One pilot’s QAR event inquiry and statistics

Full size image

The user can not only inquire the relevant event information based on the entered information of captain, flight date, flight number, aircraft type, and event type, but also evaluate the pilot’s operation performance by entering one or a group of parameters. During the system daily operation, it will notify the user for subsequent processing if a warning occurs on the system.

5 Conclusions

The Flight Operation Performance Evaluation System was introduced in this study. The system has been tested in the flight quality control department of an airline, and it will be put into use. Flight performance evaluation experts set several evaluation indexes for combination to carry out the pilot’s flight operation performance, recommendations for improvement, and other operations. The trial results showed the following:

  1. (1)

    The system can accomplish all basic functions, from the input of basic information and parameters to the output of evaluation results. It achieved QAR event inquiry and statistics, evaluation of the pilot’s operation performance, training and upgrade program, warning and other functions earlier, indicating that the integrity of the system is good.

  2. (2)

    The system provides a support tool for flight operations quality monitoring and flight training management. The system can evaluate the pilot’s operation performance from multiple dimensions, and that is more objective, effective, and reasonable. It will give an warning earlier and suggestions for improvement so that we can arrange follow-up training for the pilot. The airline’s performance rewards and punishments would get a more accurate and objective basis from this system.

  3. (3)

    The system not only provides actual data support for flight operation department to monitor flight risk, but also provides effective basis and reference for flight training department to arrange targeted improvement training. However, the system needs to be improved for shortening its response time and processing. However, the system needs some improvement to connect with the other systems of the airline for data sharing. So that we can manage the pilot’s operation performance more comprehensively and effectively.