Keywords

1 Introduction

The concept of usability arises from the internet industry in North America and some developed countries in Europe at the end of 1980s, now it has become one of the important research contents in the field of man-machine interaction by now. The international standards ISO9241 defines the usability as Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

The cockpit is the terminal for pilot and aircraft interaction directly. Just like other interactive devices, the cockpit’s usability is directly related to the flight safety. 80 % of the information in flight is provided by the display system, and the information on the display system is numerous and complicated, especially in emergency, various visual or auditory alarm signals appear at the same time. There is no doubt that the pilot will bear great psychological pressure, not only may slow down the effective operation, may also make errors in busy. So the usability design of cockpit display system should be effectiveness, efficiency and satisfaction.

Wang Haibo from Southeast University (Nanjing, P.R. China) made an eye movement experiment to evaluate the usability of the fighter cockpit digital interface. By analysing four kinds of quantitative data of the eye movement which include the pilot’s gaze point coordinate, the gaze point number, the pupil frequency and the event time of each layout, they confirmed the optimal layout since all the eye movement’s date of this one is better than all others.

Besides, Wang Haibo made another experiment to compare the digital interface’s usability of F18 and a new 4th-generation fighter. They offered the cockpit digital interface prototypes by computer simulating, and then noted down the task complete ratio, errors, task time and assists in the same task. As a result, all the four data of the new fourth-generation fighter were better than the F18’s. So they drawed a conclusion that the new fourth-generation fighter digital interface’s usability was higher.

There are at least two methods to layout the information on the display. One is Multi Criteria Decision Making (MCDM) that is employed as a quantitative tool. Firstly, experts give a importance’s sequence of all information, then according to the importance’s sequence of all positions on display, put each information onto the corresponding position. Another is Card Sorting that is used as a qualitative tool for the same problem. Firstly, each information is written on one of the cards, then users put all cards with information onto the positions of the display, explaining every choice’s reason, finally, confirm the information’s layout on the display on the basis of these reasons recorded. We can evaluate a certain layout’s usability of the information on the display according to the above methods’ result.

Wei Hengyang from Beihang University think the situation awareness (SA) is a very important factor for the aircraft cockpit display interface (CDA)’s design. Based on the simulation environment, they carry out a human-in-the-loop experiment to measure the SA by the situation awareness global assessment technique (SAGAT) and find the SA can serve as an objective way to evaluate the design of CDI. That is to say, the SA will affect the CDA’s usability.

Williams and Ball from Office of Aerospace Medicine conducted a study to assess the impact of advanced navigation displays on instrument flight procedures for general aviation, single pilot operations and found the advanced display in flight performance was advantaged under high-workload conditions. So the advanced display has a better usability under high-workload conditions.

The above researches just focus on someone aspect’s usability of the display system, such as the information layout, instead of the whole display system, in addition, the above evaluation methods have a same latent shortcoming: It can’t confirm which one is the best when there is no one group whose all date are better than others. So this paper is devoted to solve these problems, we focus on the usability evaluation of the whole cockpit display system, including information representation, information layout and information updating and so on. Differently in the method, this paper adopt Analytic Hierarchy Process to confirm the weight of each factor and its indicators, and then integrates the usability score and corresponding weight of every indicator to give a overall usability score of the cockpit display system, thus avoiding the above shortcoming.

2 Methodology

  1. A.

    Evaluation factor and its indicators

    • Factor1: The layout rationality of the whole display system

    • Its indicators: installation angle, importance principle, common use principle, function principle, sequential use principle

    • Factor2: The layout rationality of the information on display

    • Its indicators: importance principle, accordant motion principle, adjoin interrelated information principle

    • Factor3: Comprehensibility

    • Its indicators: icon simplicity, icon figurativeness, icon distinction

    • Factor4: Visibility

    • Its indicators: resolution, brightness, icon size, color settings, signal reaction time, visual angle

    • Factor5: Information updating rate

    • Its indicators: PFD, ND, ECAM/EICAS

Some annotations are as follows:

Importance principle: put the most important display in the most convenient location for the pilot.

Common use principle: put the display of highest use frequency in the most convenient location for the pilot.

Function principle: put displays which have relevant function together.

Sequential use principle: the layout of displays should be coincident with their usual use sequence in the task execution.

Accordant motion principle: when the pilot manipulates the aircraft during the flight, all the pilot, the aircraft and the aircraft’s icon on the display should be accordant in the direction of motion.

Adjoin interrelated information principle: the principle of proximity to the specific processing of the structure of the display problem. Generally should be related to the information displayed in space close to. Relevance or similarity of the display depends on the correlation of the task and the system. Therefore, the related display components should be placed together with the integration of the system.

Signal reaction time: it reflects pixel points’ response speed to input signals of the liquid crystal display, the smaller the signal reaction time is, the better the picture’s changing-over effect will be.

Visual angle: the maximum angle that one can clearly see all the contents of the screen from different directions.

  1. B.

    Evaluation procedures

    1. (1).

      Adopt Analytic Hierarchy Process to confirm the weight of each factor and its indicators. That is to say, every user fills in a judgment matrix of factors, and then we take the mean as a final factors’ weigh. In the same way, we obtain each factor’s indicators’ weighs.

    2. (2).

      Establish a set of numerical indicator evaluating system whose score is from 1 to 9 that corresponds to extremely bad, very bad, bad, slightly bad, neutral, slightly good, good, very good, extremely good respectively. And then users score the indicators according to their performance in use. This paper take the mean score as a final evaluation result.

    3. (3).

      Integrate some factor’s all indicators’ usability scores and corresponding weights to give a usability score of the factor. As such, we can obtain other factors’ usability scores. And then integrate all factors’ usability scores and corresponding weights to give a overall usability score of the cockpit display system.

3 The Usability Evaluation Experiment of a Cockpit Simulator’s Display System

  1. A.

    Participants

There are 5 participants including both undergraduate and graduate students from Northwestern Polytechnical University in this test. Among which there are one woman and four men students. They are 23.2 years old in average, ranging from 22 to 25. All the participants have plentiful experience of using computer, but didn’t familiar with the cockpit display system, so they can evaluate its usability by their experiences without mindset. That is to say, they are appropriate participants.

In this experiment, we only employ 5 users, because the key point of this article is to provide a method, of course, the more the user’s number is, the better.

  1. B.

    Apparatus

The test is provided with a cockpit simulator, which is a simplified version, but the basic functions on the display are complete (Fig. 1).

Fig. 1.
figure 1

The cockpit simulator

Full size image
  1. C.

    Tasks

In order to exhibit the display system’s performance in work, we will manipulate the plane in the simulator take off, cruise and land. Experimenters look at the display carefully, fill the judgment matrix of factors and the ones of each factor‘s indicators, and give their usability scores to all the indicators.

  1. D.

    Results

The experimenters are marked (A), (B), (C), (D), (E) respectively. Due to the experimental data is massive, we only list some of the data as follows (Table 1):

Table 1. A’s judgment matrix of factors
Full size table

We can work out A’s factors’ weights by using MATLAB after putting the above judgment matrix into computer, the result is as follows (Table 2):

Table 2. A’s factors’ weights
Full size table

We can obtain B, C, D and E’s factors’ weights in the same way, the results are not listed. The following is the average of all the participants’ factors’ weights (Table 3).

Table 3. The average of factors’ weights
Full size table

Similarly, we can obtain each factor’s average weights of corresponding indicators as follows (Tables 4, 5, 6, 7 and 8):

Table 4. Factor(1)’s average weights of indicators
Full size table
Table 5. Factor(2)’s average weights of indicators
Full size table
Table 6. Factor(3)’s average weights of indicators
Full size table
Table 7. Factor(4)’s average weights of indicators
Full size table
Table 8. Factor(5)’s average weights of indicators
Full size table

We can acquire each factor’s indicators’ scores with the same processing procedure, and this paper just take the factor(1) for a example (Table 9).

Table 9. Factor(1)’s indicators’ scores
Full size table

We can figure out the factor(1)’s usability score since its each indicator’s weight and each indicator’s score is known (Table 10).

Table 10. Factor(1)’s usability score
Full size table

Similarly, we can obtain all the factors’ usability scores. Finally we’ll work out the cockpit simulator display system’s usability score (Table 11).

Table 11. Cockpit simulator display system’s usability score
Full size table

Thus it can be seen the cockpit simulator display system’s usability score is 7.46 (the full mark is 9), which is between good and very good.

4 Conclusion

This paper proposes five evaluation factors and a series of evaluation indicators, and build a set of cockpit display system’s usability evaluation method. Moreover, we verify this method is feasible by a experiment. And our work will be the first time in the history of the whole cockpit display system’s usability evaluation, undoubtedly, it will also accelerate the development of the whole cockpit display system’s usability rapidly. Similarly, we can also generalize this method to the whole cockpit or even the whole aircraft’s usability evaluation.

We will continue to optimize this method by regulating the evaluation factors and indicators with the development of the cockpit display system’s technology.