1 Introduction

Head-Worn Display System (HWDs) is an important device to realize virtual-real fusion display. According to the working principle, it can be divided into two types: optical transmission and video transmission [1, 2]. A HWD system typically is mainly composed of the source of image display system, positioning sensor system, optical imaging system, computing system and shell components. With the help of glasses or helmet, the display system is fixed on the user’s head, and presenting a complete dynamic image in front of the user’s eyes to create a feel of immersion and interaction [3, 4]. For a long time, in order to enhance the ability of interaction between pilots and cockpits, and improve the performances for flight, scholars in the aviation field are keen to apply the HWD of optical transmission to the new concept design of cockpit, so that the primary flight information and the real view of pilot can be combined into one and moving along with the movement of pilot’ head, which can realize all-round observation and improve situation awareness for pilots [5,6,7].

With the help of combined glasses, the HWD system applied in the cockpit collects the information of the real situation and measures the movement of the pilot’s head by sensing system, and, stores and processes the data through the computing system to make tracking and registering combined with the data of head posture. Then, the optical lens and the image source are used to fuse the information of characters, icons, images, alarms and other information of flight into the external view, which present in front of the pilot’s eyes to realize the interaction between the pilot and the situation around. Researches [7, 8] show that pilots using HWD often suffer from eye swelling, dizziness, nausea and difficulty in distinguishing and reading displayed information, which makes an influence on pilots’ performance in the status of comfort, efficiency and safety. Therefore, it is an indispensable thing to evaluate the ergonomics of HWD before putting it into use. At present, there is no a thorough integrated evaluation index system for the ergonomics of HWDs. This paper puts up an integrated ergonomics evaluation index system of HWDs from the way of meeting the three requirements of comfort, efficiency and safety, which provides an important reference for further evaluating whether the design and application of HWDs can meet the ergonomics requirements.

2 Method

2.1 Preliminary Index System

Through literature research and analysis of the structure and working principle [4, 8, 9], it is found that the HWD worn on the pilot’ head makes a combination of reality and virtuality and the transmission of information into the pilot’ eyes by spectacle display. In this way, the comfort of man-machine interaction, the efficiency of display and control, as well as the safety of design all make an influence on the ergonomic of HWD system, which decides whether the pilots can complete the flight task comfortably, efficiently and safely. According to relevant reference with GJB 4052, GJB 1062A, GJB 301 and so on [10,11,12,13,14], it is found that the main factors including operation of wearing, weight and center of gravity, degree of tightness, mode of transmission, components of structure and visual representation will have impact on whether pilots can complete task in a comfortable state. The main factors including quality of brightness, angle of observation, distance of exit pupil, field of view, coding of icon, coding of character, coding of color, coding of distribution, mode of interaction and time of delay will affect the speed of obtaining information for pilots, which determines the efficiency of completing missions. And the main factors including protection of head, protection of eyes and design of quick detachment will have impact on whether pilots can work in a safe condition. Therefore, a preliminary evaluation index system for ergonomics of HWDs can be established on purpose by taking the comfort, efficiency and safety as the first-level indexes, and taking its main influencing factors and detailed factors as the second-level and third-level indexes respectively, which contains 3 first-level indexes, 19 second-level indexes and 49 third-level indexes actually.

2.2 Integrated Ergonomics Evaluation Index Method

Delphi [15] is the most representative and authoritative method in the construction of index system because of its goodness of making full use of the wisdom and experience of experts in the field. In order to ensure the scientificity and rationality of the index system established, this paper implements the analysis and selection of all the indexes based on the modified Delphi method, as described next.

  1. (1)

    Organize a coordinating group

The coordinating group is composed of three members including one professor and two postgraduates to mainly complete the work of explaining the terms of indexes, determining the group of the experts, making questionnaires, organizing consultation and collecting data.

  1. (2)

    Identify consulting experts

Experts who have certain professional knowledge and rich practical experience, and are willing to participate in the implementation of Delphi method are the key to success. Members of experts in this paper are all researchers with rich experience in ergonomics, including 2 doctoral students and 10 master students.

  1. (3)

    Contents of consultation

Consultation questionnaires and questionnaires are made by the coordinating group according to each round of consultation, and the importance and operability of the all levels indicators are scored with the 5-point Likert scale by experts. The influence levels indicated by 1 to 5 were “not”, “general”, “average”, “very” and “great”. At the same time, the level of judgment Ca (theoretical analysis 0.9, practical experience 0.7, peer understanding 0.3, intuition 0.1) and familiarity Cs (0.9/0.7/0.5/0.3/0.1) of experts should be evaluated by their own to determine the authority Cr of experts.

  1. (4)

    Screening criteria

On the basis of above, we collect the data and result of consultation each round, and evaluate the degree of authority for experts, and the degree of coordination and concentration for experts’ opinions respectively by statistical analysis.

  1. ①

    Degree of authority for experts

The authority of experts is expressed by the coefficient Cr, which is generally determined by two factors: Ca, the basis on which experts make their judgment on the problem, and Cs, the familiarity of experts with the index. There are:

$$ C_{r} = \frac{{\left( {C_{a} + C_{S} } \right)}}{2} $$
(1)

Where, \( C_{r} \ge 0.7 \) is an acceptable value. The degree of authority for experts has a certain functional relationship with the prediction accuracy, and the prediction accuracy increases with the improvement of the degree of authority of experts generally.

  1. ②

    Degree of coordination for experts’ opinions

The degree of coordination of experts’ opinions indicates whether there are big differences in the evaluation of each index among the experts in the study, which is reflected by the coefficient of variation \( V_{j} \) and the coefficient of coordination \( W \).

$$ V_{j} = \frac{{\delta_{j} }}{{M_{j} }} $$
(2)

Where, \( M_{j} \) denotes the arithmetic average value of index j, \( \delta_{j} \) denotes the standard value of index j. The smaller the \( V_{j} \), the higher the degree of coordination of experts, if \( V_{j} > 0.25 \), it is recognized that the degree of coordination of experts is not enough.

$$ W = \frac{12}{{m^{2} \left( {n^{3} - n} \right) - m\mathop \sum \nolimits_{i = 1}^{m} T_{i} }}\sum\nolimits_{j = 1}^{n} {d_{j}^{2} } $$
(3)

Where, \( n \) denotes the number of indexes, \( m \) denotes the total number of experts, \( T_{i} \) is the correction coefficient, \( d_{j} \) is the difference of the arithmetic average value between index j and the sum of all index. Whether the evaluation opinions of experts have great differences is reflected by \( W \), which can be tested by Kendall W test with SPSS. Taking the significance level (p < 0.01) as the test standard, it is believed that the test value >0.01, the coordination degree of experts’ opinions on the indexes meets the requirements.

  1. ③

    Degree of concentration for experts’ opinions

The degree of concentration of experts’ opinions on the relative importance of each index is generally expressed by the arithmetic mean \( M_{j} \), median and full mark frequency \( K_{j} \).

$$ M_{j} = \frac{1}{{m_{j} }}\sum\nolimits_{i = 1}^{m} {C_{ij} } $$
(4)

Where, \( m_{j} \) denotes the number of experts, \( C_{ij} \) denotes the evaluation value of expert i for indexes j. \( M_{j} > 3 \) means that experts’ recognition of the importance of the index meets the requirements.

$$ K_{j} = \frac{{m_{{j^{\prime}}} }}{{m_{j} }} $$
(5)

Where, \( m_{{j^{{\prime }} }} \) donates the number of experts given full marks. The higher the \( K_{j} \), the higher the experts’ recognition of the index generally.

2.3 Calculation of the evaluation index weight

The weight of index is an accurate description of the impact degree between the indicators and the target, which determines the effectiveness of the results for evaluation. G1 method is an improvement of AHP to get the weight quickly and accurately. In this paper, the weight of each index is determined based on G1, as described next.

  1. (1)

    Determine the set of evaluation index

Note that \( U \) is the set of index which is composed of the first-level evaluation index \( u_{i} \). \( U_{i} \) is the set of index which is composed of the second-level evaluation index \( u_{ij} \). \( U_{ij} \) is the set of index which is composed of the third-level evaluation index \( u_{ijz} \). There are:

$$ U = \left\{ {u_{1} ,u_{2} , \ldots \;u_{i} } \right\} $$
(6)
$$ U_{i} = \left\{ {u_{i1} ,u_{i2} , \ldots \;u_{ij} } \right\} $$
(7)
$$ U_{ij} = \left\{ {u_{ij1} ,u_{ij2} , \ldots \;u_{ijz} } \right\} $$
(8)

Where, \( i \) donates the number of first-level indexes, \( j \) donates the number of second-level indexes, \( z \) donates the number of third-level indexes.

  1. (2)

    Organize experts and determine its credibility

In order to determine the credibility of experts, the background of education (\( a_{t} \)), the basis of evaluation (\( b_{t} \)), the understanding degree of the problem (\( c_{t} \)), and the confidence of the evaluation (\( d_{t} \)) for each experts is comprehensively evaluated by the Expert Comprehensive Assessment Form. Assuming that the number of experts is \( m \), and the self-evaluation value of the t-th expert is \( G_{t} \) (\( G_{t} = a_{t} \cdot b_{t} \cdot c_{t} \cdot d_{t} \)), and the credibility of the t-th expert is:

$$ R_{t} = \frac{{G_{t} }}{{\sum\nolimits_{t = 1}^{m} {G_{t} } }},\;\;t = 1,2,3, \ldots ,m $$
(9)

Then, the experts’ confidence vector is determined as \( \varvec{R} = \left( {R_{1} ,R_{2} ,R_{3} , \cdots R_{m} } \right). \)

  1. (3)

    Calculate the single weight of index

The weight questionnaires are full in by each experts to complete the importance ranking of sub-indexes under each set of indexes and the comparison and assignment of importance (defined \( r_{i} \)) between indexes \( u_{i} \) and \( u_{i - 1} \)

$$ w_{n}^{*} = \left( {1 + \sum\nolimits_{k = 2}^{n} {\prod\nolimits_{i = k}^{n} {r_{i} } } } \right)^{ - 1} $$
(10)
$$ w_{i - 1}^{ * } = r_{i} w_{i}^{ * } ,\;\;i = n,n - 1, \ldots ,2 $$
(11)

Through Eqs. (10–11), the relative weight coefficient of each index under its set can be determined and the vector \( W^{*} \) can be obtained.

$$ W^{ * } = \left\{ {w_{1}^{ * } ,w_{2}^{ * } , \ldots ,w_{n}^{ * } } \right\} $$
(12)
  1. (4)

    Calculate the integrated weight of index

The integrated weight of each index can be obtained by combining the single weight of index from every expert and its credibility. Taking the single weight of the index i evaluated by the t-th expert is \( w_{{_{i} }}^{{t^{ * } }} \), then the integrated weight of index i is determined as \( \varvec{w}_{\varvec{i}} \), there is:

$$ \varvec{w}_{\varvec{i}} = \sum\nolimits_{t = 1}^{m} {w_{i}^{t * } R_{t} } $$
(13)

2.4 Implementation of method

12 experts including 2 doctoral students and 10 master students major in ergonomics participated in this research. They were informed of the purpose and main contents of the modified Delphi method and G1 method so that all works could be performed better.

Firstly, based on the modified Delphi method, each expert was required to analyze the importance and operability of each indexes in preliminary system to the ergonomic of HWDs. According to the Consultation Questionnaires, experts evaluated the influence of the indexes with a score of 1–5, and the level of judgment and familiarity with a score of 0.1–0.9. The indexes with high scores by screening criteria were retained and the indexes with low scores were deleted, then the final integrated ergonomics evaluation index system was established by several round of consultation until experts are unanimous.

After that, based on the G1 method, each expert was required to compare and sort the sub-indexes under each index set by its importance to HWDs and evaluated the assignment between index \( u_{i} \) and \( u_{i - 1} \) with a score of \( r_{i} \) (in Table 1) according to the Weight Questionnaires, one the other hand, the credibility of each expert was also evaluated with a score of 8–10 by the Expert Comprehensive Assessment Form. Then, the integrated weight of each index were calculated by formula 13.

Table 1. The reference for the value of \( r_{i} \)
Full size table

3 Result and Discussion

3.1 Results of Consultation for Delphi

The first round of consultation evaluated the importance and operability of 3 first-level indexes, 19 second-level indexes and 49 third-level indexes under the preliminary index system to the ergonomics of HWDs. The evaluation data were tested by Kendall W from SPSS (importance: P = 0.384, operability: P = 0.301), which indicated that experts had a high degree of coordination and authority on the proposed indexes, so the evaluation results of this round were in high credibility.

Taking the \( M_{j} \) of importance ≥3, the \( M_{j} \) of operability ≥3, and the \( V_{j} \) ≤ 0.3 were used as screening criteria. The results showed that 3 first-level indexes and 19 second-level indexes were all in conditions. In the third-level indexes of comfort, the importance and operability of “ease of wearing” (\( M_{j} \) = 2.750, 2.583) and “normalization of parts” (\( M_{j} \) = 1.750, 2.667) did not meet the screening requirements because of its weaker impact on comfort, the importance of “transmission wired/wireless” (\( M_{j} \) = 2.250) as well as the operability of “coordination of component” (\( M_{j} \) = 1.583) also did not meet the requirements, which were decided to be deleted in the next round of consultation therefore. The evaluation results of the three-level indexes of efficiency showed that “types and quantities of interaction” did not meet the importance requirements (\( M_{j} \) = 2.750), so it was decided to delete. The three-level indexes of safety all met the screening requirements of importance and operability and were retained.

Combining with the expert’ opinions of the first round, the importance and operability of the remaining 3 first-level indexes, 19 second-level indexes and 44 third-level indexes to the ergonomic of HWDs were re-evaluated in the second round of consultation. Test of Kendall W (importance P = 0.314, operability P = 0.260) showed that experts’ opinions on the indexes were consistent and had high credibility. The results showed that all indexes retained were all in conditions of importance and operability. Moreover, in the three-level indexes, “compression on head and ear” (\( M_{j} \) = 4.583), “compression uniformity on head and ear” (\( M_{j} \) = 4.417) and “fit of display size and head size” (\( M_{j} \) = 4.167) all had a larger influence on comfort. “Coordination between brightness and illuminance” (\( M_{j} \) = 4.083), “contrast between image and background” (\( M_{j} \) = 4.333), “angle of view” (\( M_{j} \) = 4.250), “readability of height of character” (\( M_{j} \) = 4.133) and “size of total delay time” (\( M_{j} \) = 4.417) all had great influence on efficiency. And, “shock resistance of shell” (\( M_{j} \) = 3.583) and “preventive measures of eye” (\( M_{j} \) = 3.750) also affected the safety of the HWD to varying degrees.

No new index added in this round, so there was no need for the next round of consultation, then an integrated evaluation index system of evaluation for Head-Worn Display system was established successfully.

3.2 Results of weight for G1

Based on the index system, the results of index weight at all levels determined by formula (9–13) were shown in Table 2.

Table 2. The results of index weight at all levels
Full size table

Comparing the weights of the above indexes, it was obvious that “weight and center of gravity” and “degree of tightness” had a greater impact on comfort, “time of delay”, “quality of brightness” and “coding of character” had great influence on efficiency, and the impact of “protection of eyes” on safety was also clear, which was basically consistent with the above results of importance by modified Delphi method. Therefore, more attention should be paid to the impact of the above important indexes in actual design and application.

4 Conclusion

In order to establish an excellent integrated ergonomics evaluation index system of Head-Worn Display system, this paper used improved Delphi method and finally established an integrated evaluation index system with 3 first-level indexes, 19 second-level indexes and 44 third-level indexes starting from the way of meeting the requirements of comfort, efficiency and safety. On this basis, the weights of indexes at all levels were determined by the G1 method, and the influence degree was clear and definite, which provids a certain reference for the design and evaluation as well as the subsequent in-depth researches of HWDs, therefore.