1 Introduction

The concept of usability emerged in the 1970 s, derived from the field of human-computer interaction. With the development of computer, the usability has been paid more and more attention from 1980 s. Usability is an important product quality attribute which was used to evaluate whether the product is easy to use and consist of user’s needs and expectations [1]. Since then different definitions of usability have been provided, but the core is consistent that the user can use the product to meet certain needs [2]. User-centered design is the core methodology of usability engineering, which guides the design activities in all stages of the product development cycle. Usability design is a kind of user-friendly design. The designer always put the needs of users in mind and even invite the target user to participate in the design process. The usability design method can effectively improve performance and man-machine consistency, which greatly influence the success of the design. As an important measure of interaction design, usability describes the relationship between user and product. It is the evaluation method that users can use the product to achieve the final goal. The ultimate goal is user’s ability to use the system well. That is to say, usability is a measure of product’s quality from the user’s perspective.

The cockpit is an extremely complex man-machine interaction system. In order to successfully complete the mission, the pilot must obtain a variety of flight attitude information through the display system first. Therefore, the cockpit display system usability level directly determines the flight safety and flight efficiency. However, modern aircraft cockpits expose the problem of mismatches and incongruities between aircraft design systems and people. Researchers have done a lot of research to improve the cockpit display system usability and the interaction between the pilot and the cockpit.

Fayollas [3] found a way to influence usability through reliability and apply it to the cockpit interface. A new generation cockpit, based on the ARINC 611 standard, allows the crew to control the display unit using a keyboard and cursor. Then, they analyzes the impact of system fault tolerance on usability and finds a trade-off between reliability and usability under complex training tasks.

Harbour [4] explored the relationship between situational awareness and display usability. Situational awareness is the ability to sense information and act in an acceptable way, and can directly affect the display of the HUD and the HDD, resulting in a change in the difficulty of the task. Harbour proposed the basic neurocognitive factors, determined its impact on the formation of SA, and studied the display usability. Visual attention, perceptual and spatial working memory were evaluated as predictors of different task difficulty, and the data of three predictors were statistically significant with the change of task difficulty.

Yanyan Wang [5] used the eye tracking technology to evaluate the usability of the static man-machine interface of the cockpit and established the static eye movement test data model. Based on the typical aircraft cockpit display interface, the participants need to find the corresponding position in the original image in the shortest possible time according to the given target image. The results show that there are 17 indicators of significant differences in the 25 commonly used eye movement index. After factor analysis, five principal components were extracted, namely AOI, average pupil area, first glance distance and average fixed duration in AOI.

Yanbin Shi [6] proposed a cloud model to assess cockpit display system usability. The cloud model is composed of gray system theory and gray albedo function for qualitative and quantitative uncertainty transformation. They evaluated the human-computer interaction from the view of learning, efficiency, memory, error and satisfaction, and obtains the digital characteristics of each index in the cloud model. According to the model, it is possible to improve the usability of the flight simulator man-machine system by performing sensitivity analysis.

The study mentioned above has the following problems. Fayollas and Harbour explore the relationship between usability, reliability and situational awareness, but usability is not analyzed and evaluated separately. Wang Yanyan evaluates the various indicators using eye movements under static tasks, but in actual flight, dynamic display is more complex than static changes. Yanbin Shi’s selection of evaluation indexes is very incomplete and has a great influence on the accuracy of the assessment results. Therefore, this paper is devoted to cover the shortages above. The paper will analyze the basic concepts of usability, and then select typical flight scenes, and evaluate the usability indexes of the cockpit display system comprehensively.

2 Method

2.1 Evaluation Process

There are many methods to study usability, but the most basic and useful is user testing, which has 3 basic components [7]:

  1. 1.

    Get hold of some representative users.

  2. 2.

    Ask the users to perform representative tasks with the design.

  3. 3.

    Observe what the users do, where they succeed, and where they have difficulties with the user interface.

In order to assess the usability of the cockpit display system, this paper select several graduate students as subjects. Refer to the A320 Flight Manual, takeoff, climb, descent, approach and landing is select as a typical mission. The remaining question is how to obtain the test data, select a more reasonable assessment factors and find the problem in the operation.

2.2 The Selection of Evaluation Indicators

There are a lot of factors influencing the usability of cockpit display system. The appropriate evaluation factors selected is key to evaluation accuracy and the result’s effective for design. While more evaluation factors are selected, there will be increased overlap of indexes. However, if the factors are not sufficient, it would lead to the incomplete evaluation. These limitations would affect the accuracy of the evaluation and make the evaluation atypical. The challenge of the factors chosen is that it is always difficult to fully enumerate all the factors making the evaluation results not comprehensive. In addition, it is difficult to construct a quantitative evaluation model directly, which will weaken the guidance to design.

Considering the usability indicators of cockpit display system are numerous and difficult to quantify, this paper try to use the questionnaire to collect test results. Comparing and screening the usability dimensions defined by the current standards such as ISO 9126, ISO 9241 and ISO 13407, indicators relating to cockpit display system usability are selected as shown in Table 1.

Table 1. Usability indicators of cockpit display system
Full size table

At the same time, the following design factors are summarized according to the cockpit display system’s own characteristics, as shown in Table 2. The design factor and the usability index are combined to construct the questionnaire for collecting the test results. Each small problem of the questionnaire is the evaluation factor.

Table 2. Design factors of cockpit display system
Full size table

2.3 Factor Analysis and Multiple Linear Regression

Factor analysis is used to combine and classify the factors which overlap each other, and obtain the factors that are not related to each other. The essence of factor analysis is to reduce the number of significant variables, the main mathematical thought is to reduce dimension and simplify, and the main purpose is to find a small number of essential factors. The calculation procedure of the factor analysis method is as follows:

  1. 1.

    Normalization of raw data: Make the data positive and non-dimensional

  2. 2.

    Determine the number of common factors: to obtain the correlation coefficient matrix and its eigenvalues and eigenvectors, the contribution rate of accumulation

  3. 3.

    Solving the factor load matrix: the meaning of each element in the factor load matrix represents the similarity coefficient between the corresponding variable and the common factor, reflecting the relative importance between them. The higher the absolute value is, the higher the correlation degree is.

  4. 4.

    Rotation of the factor load matrix: simplify the structure of the factor load matrix

After determining the final evaluation factors, the paper use multiple linear regression analysis to determine the weight coefficient. Multivariate linear regression analysis is a mathematical statistical method to deal with the statistical correlation of variables. Multivariate Regression Assume that the dependent variable is a multivariate linear function of the independent variable, and the stepwise regression method is used to screen these independent variables to find the statistical multiple linear regression model. The basic idea of multivariate regression analysis is that although there are no deterministic functional relationships between multiple independent variables and dependent variables, the mathematical expression that best reflects their relationship can be found. An important use of multiple linear regression analysis is to interpret and predict the data, and to calculate the accuracy of interpretation and prediction.

3 Usability Testing

3.1 Tasks

During the course of the experiment, the participants need to complete the whole process of the aircraft driving from takeoff to landing, and then fill out the questionnaire according to the subjective experience of flight test. The questionnaire included 35 declarative sentences, and the participants were evaluated according to nine levels from “very dissatisfied” to “very satisfactory”.

3.2 Participants

There are nine testers involved in the experiment and all of whom are graduate students who have been studying usability for one or two years. They are between 22 and 25 years old, with an average age of 22.8 years, so we can assume that they are appropriate participants. Although the participants have obvious deficiencies in the control and control ability compared with the pilot, they have the ability of cognition and judgment on the cockpit human-computer interaction interface. The satisfaction questionnaire was conducted under the guidance of the main test, which ensured the validity of the questionnaire.

3.3 Apparatus

The test is provided with a cockpit simulator, which is a simplified version, but the basic functions on the display are complete (Fig. 1).

Fig. 1.
figure 1

The cockpit simulator

Full size image

3.4 Results

As the amount of experimental data is large, the original survey data is no longer listed here. The scores of each index in the questionnaire were statistically analyzed, and the cumulative contribution rate was obtained by factor analysis using PASW in Table 3.

Table 3. The cumulative contribution rate of each factor
Full size table

As can be seen from the table, there are three main indexes affecting the usability of the cockpit display system, whose cumulative contribution rate reaches 86.584%. And the rotation component matrix is obtained in Table 4 after the rotation of the factor load matrix.

Table 4. Rotation component matrix
Full size table

In the rotation component matrix, factors whose load is less than 0.5 are deleted. And the three main indexes that affect the usability of the cockpit display system are finally determined through summarizing the rest of the factors in the table. The final three main factors are satisfaction, learning and error.

Multivariate regression analysis can determine how cockpit display system usability is affected by factors in the model. Three main factors of the model are independent variables while cockpit display system usability is the dependent variable. The results of the regression analysis are shown in Table 5.

Table 5. The weight of the three factors
Full size table

Take satisfaction for example, comparing its regression model with the evaluation factors can finally find design factors which influence satisfaction. The results are shown in Table 6.

Table 6. Comparison table of design factors
Full size table

4 Conclusion

The evaluation model proposed in this paper is verified. Three main factors influencing the usability of the cockpit display system are proposed, and the evaluation index system is established and a better evaluation result is obtained. Furthermore, this paper finds the design factors behind the indexes, and the problems in the cockpit display system design are found. Moreover, the evaluation model established in the paper can be applied to the whole cockpit usability assessment. This model can also be extended to other man-machine interface usability assessment, and promote the development of display system interface usability.

With the development of cockpit display system technology, we will continue to improve the experimental program, adjust the evaluation factors and indicators to optimize the evaluation model to get a more accurate assessment results.