Keywords

1 Introduction

Human factors engineering is important and has been brought into the regulations for the operation of nuclear power plants, such as NUREG-0800: Standard Review Plan (NRC, Revision 2, 2007) and NUREG-0711: Human Factors Engineering Program Review Model (NRC, Revision 2, 2004). However, there is still a discrepancy between the regulations and the practices of the operation with digital human-system interfaces in the new constructing nuclear power plant in Taiwan, known as the Lungman nuclear power plant (LMNPP).

According to the NUREG-0711, Situation Awareness (SA) is one of the measures in the Verification and Validation (V&V) phase, and should be measured by state-of-the-art methods. However, current practice in the LMNPP for SA measurement is only based on observations. That is, operators are asked to make various drills under the observation of several domain experts. These experts then decide if operators’ SA is appropriately supported by the design of human-system interfaces. The main disadvantage of this approach is that it is relatively subjective, and the homogeneous background of these close-knit experts might lead to the phenomenon of groupthink. Furthermore, the approach is resource-consuming in terms of time and cost. Although the LMNPP has passed the V&V Level 3, a focus group interview on operators revealed that the SA support from current human-system interfaces was one of major operation issues in the LMNPP (Liang et al. 2008). Therefore, the aim of this study was to develop a relatively objective and recourse-effective method for SA prediction and measurement.

Before the review of related literature in the next section, a brief background about the Main Control Room (MCR) in the LMNPP is introduced as follows:

Digital instrumentation and control has employed to the design of human-system interfaces in the LMNPP. Figure 1 illustrates the layout of the MCR in the LMNPP. A Wide Display Panel (WDP) is on the front wall of the MCR, and the Main Control Console (MCC) is located between the WDP and the Shift Supervisor Console (SSC).

Fig. 1.
figure 1

Layout of the main control room

Full size image

While a schematic display of equipment and flow pipes occupies most of the WDP, system-level alarm indicators are displayed on the top strip of the WDP. These visual indicators are grouped into five adjacent rectangle areas as shown in Fig. 2. According to different abnormal situations, specific indicators would turn red and/or yellow.

Fig. 2.
figure 2

Layout of the system-level alarm display (Color figure online)

Full size image

Since the system-level alarm displays are one of the major channels for operators to diagnose abnormal situations and take further actions according to the standard Abnormal Operation Procedures (AOPs), the SA support by the system-level alarm displays is important and is the focus of this study.

2 Related Research

The most referred definition of Situation Awareness (SA) is Endsley’s three-level model: the first level is the perception on elements in a specific temporal and spatial environment; the second level is the comprehension of the meanings of these elements; and the third level is the projection of the future status of these elements (Endsley, 1988; 1995).

SA was also defined as an adaptive and externally directed consciousness (Smith and Hancock, 1995). According to the perceptual cycle (Neisser, 1976), people construct their schema or mental models of present environment based on their internal knowledge and experience. These schema or mental models may guide them to explore external environment and obtain new information. To complete the perceptual cycle, the obtained information may update the current schema and mental models. In other words, the perception process is influenced by both a top-down (goal-driven) processing and a bottom-up (stimulus-driven) processing so that attention is directed by individual’s goal and expectation as well as by novel, dynamic, or intense stimuli in the environment (Wickens et al. 2004).

The SA measurement methods can be characterized into four types: performance measures, physiological parameters, probes, and subjective ratings (Salmon et al. 2006).

For the methods of performance measures, the SA can be measured by the success rates of operator’s signal detection tasks (Gugerly, 1997). However, the performance of signal detection is also influenced by operator’s level of expertise and response bias (Green and Swets, 1966). Thus, the success rates may not be appropriate as the SA measures (Salmon et al. 2006). Nevertheless, this approach was effective for the SA comparison of different alternatives, such as measuring the SA support provided by two different user interface designs (Andre et al. 1991).

Eye tracking is a common technique to measure SA. The Areas of Interest (AOI) are recognized through the distribution of gaze. The SA for the AOI is then measured by the gaze distribution percentage on these AOI (Smolensky 1993; Ha and Seong, 2007). However, special apparatus and equipment is needed for collecting physiological data, and it may be difficult to obtain or operate these apparatus and equipment (Salmon et al., 2006).

The most popular probe technique is the Situation Awareness Global Assessment Technique (SAGAT) (Endsley 1995). The tasks are performed on simulators. With random blanks on the display screen during the tasks (freeze), operators are asked for answering SA relevant questions. Operators’ SA is then measured by the accuracy of the answers. One similar technique, Situation Awareness Control Room Inventory (SACRI) (Hogg et al. 1995), has been designed for measuring the SA in nuclear power plants. Different from “freezing” the screen, the real-time probe technique is that operators answer the questions while they perform the tasks without the “frozen” blanks, but operators’ SA is also measured base on the accuracy of the answers. Examples of this technique are Situation Present Assessment Method (SPAM) (Durso, et al. 1998), Global Implicit Measure (GIM) (Vidulich and McMillan, 2000), and SAHSA (Jeannott et al. 2003).

Compared to other methods, subjective ratings are easier to apply. Situation Awareness Rating Technique (SART) (Taylor 1990) uses ten dimensions for participants to rate the level of SA. These ten dimensions are grouped into three categories: Demands on attentional resources (D), Supply of attentional resources (S), and Understanding of the situation (U). The level of SA is measured by the sum of rating scores of U and S subtracted by the score of D. Examples of similar techniques are: Crew Awareness Rating Scale (CARS) (McGuinness and Foy 2000), Mission Awareness Rating Scale (MARS) (Mattews and Beal 2002), Situation Awareness Rating Scale (SARS) (Waag and Houck 1994), and Cranfield Awareness Rating Scale (C-SAS) (Dennehy 1997). Different from previous multi-dimensional approaches, Situation Awareness Subjective Workload Dominance (SA-SWORD) (Vidulich and Hughes 1991) uses a uni-dimensional rating to measure SA. This technique measures relative SA among a set of displays by a series of pair comparisons on a nine-level scale, similar to the procedures in Analytic Hierarchy Process (AHP) (Saaty 1980). Operators are the persons to answer the questions in all mentioned rating techniques, but one exception is the Situation Awareness Behavioral Rating Scale (SABARS) technique (Mattews and Beal 2002) in which domain experts give the ratings.

To be able to both measuring and predicting SA, few analytical models have been developed. One approach is to apply the Bayesian Belief Networks (BBNs) to predict operators’ SA in Endsley’s (1995) Level-2: Comprehension (e.g., Kim and Seong 2006; Miao et al. 1997). The other analytical model is Attention-Situation Awareness (A-SA) (Wickens et al. 2005).

Wickens and his colleagues have applied the A-SA model to predict pilots’ SA based on four parameters: Salience, Effort, Expectancy, and Value (SEEV) (Wickens et al. 2001; Wickens et al. 2005). They claimed that the probability of attention (P(A)) on an Area Of Interest (AOI) can be predicted by the Salience (S) of the objects or events in that area, the Effort (EF) of turning attention to that area, the Expectancy (EX) of the appearance of objects or events in that area, and the Values (V) of the objects or events in that area. A linear relationship, as shown in the equation below, is assumed between the P(A) and the four parameters.

$$ P(A) = sS - efEF + (exEX + vV) $$
(1)

On the right hand side of the equation, the four capital abbreviations represent the four parameters, whereas the four lowercase abbreviations represent the corresponding coefficients. The equation indicates that there are positive relationships between the probability of attention and the parameters of Salience, Expectancy, and Value, but a negative relationship between the probability of attention and the parameter of Effort. While Salience and Effort are stimulus-driven parameters, Expectancy and Value are goal-driven parameters.

The purpose of this study is to develop and validate an analytical model based on the SEEV framework for the prediction and measurement of SA on the system-level alarm displays in the Lungman nuclear power plant (LMNPP).

3 Methods

3.1 Participants

Forty university students were recruited as participants. Twenty were female and twenty were male. Their average age was 23.6 years old with a standard deviation of 2.2 years old. All reported having normal or correct-to-normal vision. Students instead of real operators were recruited because in this way we could manipulate participants’ goal-driven parameters (Expectancy and Value) in the experiment design.

3.2 Experiment Design

A multiple linear regression model with the four parameters was designed as follows:

$$ Y = \alpha S - \beta EF + \gamma EX + \delta V + \varepsilon $$
(2)

The Y was the gaze distribution percentage on any Area Of Interest (AOI) on the system-level alarm displays; α, β, γ, and δ were the four coefficients of S, EF, EX, and V, respectively, and ε was the error term.

Salience and Effort were manipulated through ten different types of system alarm displays with various intensities and locations of alarm indicators. For the parameter of Salience, a value of 0 was assigned when there was no alarm signal in the area. A value of 1 was assigned when the area appeared either red or yellow alarm signals. Finally, a value of 2 was assigned when the area appeared both red and yellow signals. For the parameter of Effort, a value of 0 was assigned to the central rectangle (labeled as Area 3). A value of 2 was assigned to the two lateral rectangles (labeled as Area 1 and 5). Finally, a value of 1 was assigned to the two rectangles in between (labeled as Area 2 and 4). The assigned values for each type of the ten alarm displays and each of the five alarm display rectangles are shown in Table 1. The corresponding list of alarm displays is illustrated in Table 2.

Table 1. Assigned values of salience (S) and effort (EF): (S, EF)
Full size table
Table 2. Corresponding list of alarm displays
Full size table

For the parameter of Expectancy, a value of 1 was assigned as the participant was instructed the area(s) that would appear alarm signals in the next alarm display, whereas a value of 0 was assigned as the participant did not be instructed. The manipulation of the parameter of Value was similar. While a value of 1 was assigned as the participant was instructed the area that was important in the next alarm display, a value of 0 was assigned as the participant did not be instructed. The frequencies of being instructed as an important area were the same for each of the five alarm display areas (rectangles). The forty participants were evenly divided into four groups with five females and five males in each group. Each group received different combinations of instructions presented in Table 3.

Table 3. Assigned values of expectancy (EX) and value (V): (EX, V)
Full size table

For data analysis, the data were randomly split by the 80–20 rule in terms of the numbers of participants and displays. The first regression model was built by the data from the 80 % of the participants (32 persons), and the model was validated with the data of remaining 20 % of the participants (8 persons). The second regression model was constructed by the data from the 80 % of the alarm displays (8 displays), and the model was validated with the data of remaining 20 % of the alarm displays (2 displays).

3.3 Apparatus and Materials

A gaze tracking system, FaceLab Version 4.5, was used to collect data in a mimic main control room (MCR) setup. As shown in Fig. 3, a picture of the Wide Display Panel (WDP) was projected to the screen. The distance between the participant and the screen has been adjusted so that the relative viewing angles and size would be similar to the real MCR condition in the LMNPP.

Fig. 3.
figure 3

Experiment setup

Full size image

3.4 Procedures

Before the experiment, participants were asked to fill up a personal information form. Then the basic scenarios of alarms in nuclear power plants were introduced and demonstrated to the participants. In the experiment, ten system-level alarm displays were randomly projected on the screen for vigilance. Each display sustained five seconds, and a black screen was displayed between the alarm displays. For the groups of participants designed to receive the instruction of Expectancy and/or Value instruction, the instruction(s) would be given during the black.

4 Results

4.1 Construct and Test by the Split of Participants

A multiple linear regression model was constructed as below based on the data from the 80 % (32) randomly-selected participants:

$$ \mathop Y\limits^{ \wedge } = 9.53 + 15.48S - 5.39EF + 2.83EX + 37.56V $$
(3)

All the coefficients and the intercept were significant at p < 0.05 level. The four parameters could account for about 78 % of the variance of the gaze distribution percentage (R2 = 0.783; Adjusted R2 = 0.779). Applying the regression model to the remaining 20 % data, this model could account for about 77 % of the variance of the gaze distribution percentage (Predicted R2 = 0.766).

For the two bottom-up processing parameters, it indicates that Area of Interest (AOI) would draw about 15 % more attention as the value of Salience has increased one unit, such as the status was from no signal to any one-color signal(s), or from one-color signal(s) to two-color signals. In contrast, the AOI would lose about 5 % attention as the value of Effort increased one unit, indicating that the two lateral AOIs (Area 1 and 5) lost about 10 % attention than the middle AOI (Area 3), and two AOIs in between (Area 2 and 4) lost about 5 % attention than the middle AOI.

For the two top-down processing parameters, it indicates that the AOI would attract about 3 % more attention as the value of Expectancy has increased one unit. That is, the status was from without the Expectancy instruction to with the instruction. On the other hand, the AOI would receive about 38 % more attention as the value of Value has increased one unit. That is, the status was from without the Value instruction to with the instruction.

4.2 Construct and Test by the Split of Alarm Displays

A multiple linear regression model was constructed as below based on the data from the 80 % (8) randomly-selected alarm displays:

$$ \mathop Y\limits^{ \wedge } = 8.29 + 17.23S - 4.96EF + 2.60EX + 40.06V $$
(4)

All the coefficients and the intercept were significant at p < 0.05 level. The four parameters could account for about 83 % of the variance of the gaze distribution percentage (R2 = 0. 830; Adjusted R2 = 0.826). Applying the regression model to the remaining 20 % data, this model could account for about 72 % of the variance of the gaze distribution percentage (Predicted R2 = 0.718).

For the two bottom-up processing parameters, it indicates that the AOI would draw about 17 % more attention as the value of Salience has increased one unit, and the AOI would lose about 5 % attention if the value of Effort increased one unit. The values of these two coefficients were very similar to the values of the corresponding coefficients in Eq. (3).

For the two top-down processing parameters, it indicates that the AOI would attract about 3 % more attention as the value of Expectancy has increased one unit, and the AOI would receive about 40 % more attention as the value of Value has increased one unit. Again, the values of these two coefficients were very similar to the values of the corresponding coefficients in Eq. (3).

5 Discussions and Conclusion

Results show that the multiple linear regression models in Eqs. (3) and (4) could successfully predict the variance of SA for the system-level alarm displays in nuclear power plants. In general, the positive relationships between the gaze distribution percentage and the three parameters (Salience, Expectancy, and Value), and the negative relationship between the gaze distribution percentage and the parameter of Effort were consistent with the SEEV framework. The parameter of Value had the highest strength of the relationship with the gaze distribution percentage in this study, followed by the parameter of Salience. However, the strengths of the relationships between the gaze distribution percentage and the parameters of Expectancy and Effort were relatively marginal.

For the two bottom-up processing parameters, the high strength of relationship between the gaze distribution percentage and the parameter of Salience suggests that operator’s SA would be enhanced by the design of alarm indicators with red and/or yellow signals. The marginal effect from the parameter of Effort on the gaze distribution percentage indicates that the alarm displays were designed within the range of easy eye rotation so that operator’s SA would not be substantially decreased from the central to the lateral display areas.

For the two top-down processing parameters, the high strength of relationship between the gaze distribution percentage and the parameter of Value suggests that operator’s SA would be significantly directed by operator’s priorities to the alarm display areas. The small effect from the parameter of Expectancy on the gaze distribution percentage indicates that operator’s SA would not be much affected by operator’s expectation on the appearance of alarm signals in display areas. Since these two parameters were manipulated in the laboratory experiment, it is worthwhile to explore the relationships between operators’ SA and their priorities and expectations of the alarm displays.

The analytical model developed in this study should be a necessary complement to current practice of SA measurement for predicting or measuring operators’ situation awareness on either current or new designs of alarm displays. Furthermore, this model can be applied to suggest the improvement of alarm display designs in a resource-effective manner, and ultimately enhance the safety of nuclear power operation.