Keywords

1 Introduction

At present, researchers have reached a general consensus that 60% to 90% of all system accidents, regardless of their field differences, can be attributed to human error. Especially in the process of power grid regulation and control, because it is not directly related to the operation of power production equipment, but the cognitive decision-making of power grid state by regulators, human factors have a particularly significant impact on the safe operation of power system.

Yerkes-Dodson rule [1] holds that there is an inverted U-shaped relationship between workload and performance. Moderate workload level can make performance reach its peak state, but too small or too large workload will reduce work efficiency, as Fig. 1 shows.

Fig. 1.
figure 1

Graph of Yerkes-Dodson law

Full size image

Appropriate workload intensity can improve the efficiency of operators, reduce the error rate of human risk, and significantly to the safe operation of the power grid.

Typical workload assessment methods include subjective evaluation, performance measurement and physiological measurement [2,3,4]. The subjective evaluation method, such as NASA-TLX, which cannot reflect the detailed situation of dispatchers at all times. Task performance measurement is based on the completion of the task, while in the grid system, he dispatching performance has delayed, because of the power dispatchers are not the actual operators of power facilities. Physiological measurement relies on the evaluation of physiological signals to assess workload. The results of this method are objective and reasonable to some extent, but it requires extended time and money.

This paper combines cognitive theory with system engineering method, uses Wicken’s and Yeh [5] multi-resource channel theory as shown in Fig. 2, which decomposes and divides the operator’s working process according to his behavior, and interprets the resource allocation relationship.

Fig. 2.
figure 2

Multiple resource model theory

Full size image

On the basis of Wicken, McCracken and Adrich [6] proposed the VACP (Visual Auditory Cognitive Psychomotor) model, which takes the occupancy of multi-channel resources as the main index to evaluate the workload, but this method ignores the accumulation of time factors in the process of task execution.

In this paper, we proposal a VACVP (Visual Auditory Cognitive Voice Psychomotor) model, which includes five resource channels: visual channel, auditory channel, cognitive channel, voice channel and psychomotor channel. This model covers physiological and psychological workload evaluation, and can describe the workload intensity of power dispatchers comprehensively. What’s more, it considers the time factor. According to Wickens [7], current workload can be judged by calculating the resource occupancy of each channel in a certain period of time. Therefore, we establish a comprehensive, continuous system suitable for serial and parallel tasks workload prediction model which based on multiple resource occupancy.

Besides, we proposal a comprehensive workload evaluation model based on physiological factor measurement to verifies the accuracy and availability of the VACVP workload prediction model. In this model, operators’ physiological factors such as voice, behavior, psychology and action are objectively evaluated by camera, micro-phones and heart rate instrument. The actual workload value is mainly calculated by PCA, including feature extraction and weight calculation.

2 Methodology

2.1 Task

Multiple channel physiological data were collected from DTS anti-accident exercise scenario and daily work scenario of dispatching hall, respectively. In the dispatching hall, the daily work of dispatchers mainly includes record and monitor devices, which lasts for a long time (6–8 h) and has a large working area. In the DTS anti-accident scenario, its main characteristics are short duration, high workspace intensity, mainly dealing with accidents, and heavy mental workload. In this data acquisition method, it can reflect the work status of power dispatchers comprehensively.

2.2 Participants

Four power grid dispatchers were selected as the experimental subjects, with an average age of 33 years old and working life at about 6 years. The experimental subjects had no physical and psychological problems. They did not take tea, coffee and other psychoactive drugs before and during the experiment, in order to ensure the natural and good state in the experimental process.

2.3 Apparatus

In this study, the comprehensive evaluation model is based on physiological measurement, which need some measuring equipment as Fig. 3 shows. The camera is used to record the video in order to get the motion indicators, which has 140° wide angle of view, 1080P resolution, 60 Hz sampling frequency and high sensitivity. The speech and visual indicators are collected by the camera built in microphone. The heart rate indicator Mio Alpha heart rate watch obtains dispatcher’s heart rate information.

Fig. 3.
figure 3

Camera, Mio alpha and Micropthone

Full size image

2.4 Experiment Design

Firstly, the experimenter installed the fixed wide-angle camera in advance, and arranged local area network to connect all measuring devices to the network and synchronize the clock. Half an hour before the start of the experiment, the experimenter wore a heart rate watch for the dispatcher, started the off-line recording function and started the wide-angle camera recording function. When all tasks were completed, the experimenter stoped recording the equipment and exported the data, so all the experimental data were collected [9].

The whole Experimental process as Fig. 4 shows. There were two models of data processing: the VACVP workload prediction model and the comprehensive workload evaluation model. In this experiment, the log file was recorded by the dispatcher based on the work plan and the physiological measurement data was obtained by the apparatuses. Two kinds of data were processed by time stamp alignment method to ensure data consistency. According to the log file, at a time point, the recorded task was decomposed into five resource channels and the workload was calculated based on the VACVP model. Meanwhile, according to the comprehensive evaluation model, the workload value is calculated by PCA to process physiological measurement data.

Fig. 4.
figure 4

The whole experimental process

Full size image

Finally, by comparing the values of predicted workload and actual workload to judge the accuracy and validity of VACVP workload prediction model.

2.5 Model

VACVP Workload Prediction Model.

In VACVP, workload is composed of mental and physical loads [8], time and resource occupancy are taken as two main calculation indicators. From the perspective of multi-resource occupancy theory and information processing, brain load includes visual (V) auditory (A) and cognitive (C) in multi-resource occupancy theory, mainly in the information acquisition and information processing stage [7]. Physical load is related to human operation and movement, mainly includes psychomotor (P) and voice (V), focusing on the operational response stage [5, 7]. The whole process of task execution is described by information acquisition of VAV, information processing of C and motion P as response [7]. Each task type of the operator is assigned to five channels of VACVP in the multi-resource occupancy theory. Different channel occupancy weights are given according to the utilization of resource channels involved in task types. The proportion of channel resources occupied by each task was assessed by task analysis experts combined with VACVP rating scale, as shown in Table 1.

Table 1. VACVP rating scale
Full size table

Furthermore, the workload of a task L in the same time period is weighted by the workload of each channel.

$$ {\text{WL}} = WL_{V} + WL_{A} + WL_{C} + WL_{V} + WL_{P} $$
(1)

Where: \( WL_{v} \), \( WL_{A} \), \( WL_{C} \), \( WL_{V} \), \( WL{}_{P} \) mean the workload of the channel Visual, Auditory, Cognitive, Voice and Psychomotor.

At a time point, if there are n tasks that operating simultaneously, the workload of serial and parallel tasks is added up by the workload of each task.

$$ {\text{W}}L_{total} = WL_{task1} + WL_{task2} + WL_{task3} + \ldots \ldots + WL_{taskn} $$
(2)

The corresponding relationship between VACVP evaluation and workload level is shown in Table 2.

Table 2. Mapping relationship between VACVP evaluation values and ranks
Full size table

The conversion formula for mapping the weighted score in Table 2 to the corresponding workload level is as follows.

$$ {\text{y}} = 20 \times \left( {\frac{1}{{1 + 3^{ - 0.1x} }} - 0.5} \right) $$
(3)

Where, x represents weighted score and y represents workload level.

Comprehensive Workload Evaluation Model.

In this model, every measurement is distributed. Each measuring device synchronizes the internal clock through the LAN before each test, and then carries out the off-line measurement independently. Record the time stamp accurately to the millisecond level at each sampling point in the measurement process. After each recording is completed, the data files collected by the sensors are readed through the time axis synchronous calibration method [10], and the data is time aligned and integrated.

In this model, the dispatcher’s behavior is analyzed by video processing. The motion is detected by the combination of skin color test and motion test. Firstly, the two frames \( f_{n} \), \( f{}_{n - 1} \) in the video sequence are subtracted, and the absolute value of the difference image is taken to get the corresponding difference image \( D_{n} \).

$$ {\text{D}}_{\text{n}} \left( {x,y} \right) = \left| {f_{n} \left( {x,y} \right) - f_{n - 1} \left( {x,y} \right)} \right| $$
(4)

Then, the threshold \( T \) binarization of \( D_{n} \) is processed, the connected row analysis is carried out to obtain the image contour \( R_{n} \), which contains the complete moving object.

$$ R_{n} \left( {x,y} \right) = \left\{ {\begin{array}{*{20}l} {255,} \hfill & {D_{n} \left( {x,y} \right) > T} \hfill \\ {0,} \hfill & {else} \hfill \\ \end{array} } \right. $$
(5)

At the same time, the skin color of the image is checked and processed to get the connected area with human skin color. The region correlation comparison is carried out on the \( R_{n} \) image to get the final target detection area and the dispatcher’s motion index information. The motion results obtained from the above processing are shown in the following Fig. 5.

Fig. 5.
figure 5

Movement result

Full size image

According to the different frequencies of human voice and ringtone, the upper and lower limit of cut-off frequency of voice signal passband is 100 Hz, 400 Hz, and the upper and lower limit of stop-band cut-off frequency is 50 Hz and 850 Hz respectively. The voice signal and ringtone are separated, the short-term energy and spectral entropy of voice are extracted, so that the voice index information is obtained. The speech energy and speech entropy results obtained from the above processing are shown in the following Fig. 6.

Fig. 6.
figure 6

Speech energy and speech entropy

Full size image

The collected heart rate signals are filtered to eliminate the interference of external conditions, remove the extreme value of heart rate information interval, and retain the data of heart rate signals between 50 and 150. The heart rate results are shown in the Fig. 7.

Fig. 7.
figure 7

Heart rate

Full size image

After the above data processing, the comprehensive evaluation model can be mathematically set up. The workload of comprehensive workload evaluation model can be calculated by the following equation:

$$ W = \beta_{1} M + \beta_{2} H + \beta_{3} S $$
(6)

Where:

  • Movement, Heart Rate, Speech energy are the results of physiological parameters stated above.

  • \( \beta_{ 1} ,\beta_{ 2} ,\beta_{ 3} \), are the weights to represent the contributions of movement, heart rate and speech energy. They are set by the algorithm called PCA (Principal Component Analysis).

In this paper, PCA is used to quantity the contributions of these factors (movement, heart rate, speech energy, speech entropy). The essence of PCA is the process of transforming the high-dimensional space into low-dimensional space, which makes the problem become more intuitionistic and simple [11]. Based on the above PCA processing, the \( \beta \) values of comprehensive evaluation model can be worked out as following equation:

$$ W = 0.6902M + 0.1637H + 0.1379S $$
(7)

3 Result

3.1 Result of VACVP Workload Prediction Model

According to the VACVP model, the weighted scores of each channel in the same time period and the total forecasting workload level of the whole model can be obtained by processing the data of the task planning timeline. the results of VACVP model are as following.

The results in Fig. 8 clearly reflect the occupancy of every resource channel and the change of workload level during the execution of tasks. In particular, it is pointed out that the number in the longitudinal axis of the task time line graph represents the number of the current type of work, rather than the sum of the number of task types. If the task type occurs at a certain time, then a box appears at the number representing the task type, and the sum of the number of vertical blocks represents the total number of parallel task types.

Fig. 8.
figure 8

Result of VACVP workload prediction model

Full size image

3.2 Result of Comprehensive Evaluation Model

According to the comprehensive workload evaluation model, the results of actual workload level are as shown in Fig. 9.

Fig. 9.
figure 9

Result of comprehensive workload evaluation model

Full size image

In Fig. 9, the meanings of parameters in the task time line graph are the same as that in Fig. 8. According to Fig. 9, it can be concluded that the size of the workload is related to the complexity of a single task and the number of task types. When the complexity of a task is greater, its workload will increase. For example, when time = 3000 s and the task type is 11, the workload level is 5 (actually task 11 is indeed a complex task); when the number of task types is more, its workload will also increase. For example, when time = 14000 s, the sum number of tasks is 3, the workload level is 6.

3.3 Result Comparison

By showing the two workloads in the same time period, we can compare the values between the two models, the comparation results are as Fig. 10 shows.

Fig. 10.
figure 10

Comparation of two models’ results

Full size image

From Fig. 10, we can see that the real workload and the forecast workload are positively correlated with the change of task type and total task amount in a certain period of time. This meanings the two workload measurement models are both effective and applicable to the actual working conditions.

By analyzing the changes of the two results data, it shows that the trend of workload forecast value is basically consistent with the actual value. While, the real value is slightly larger than the forecast workload value. Specific data results are shown in Table 3.

Table 3. Results comparison
Full size table

Table 3 shows the data of predicted and actual workload levels at specific time points, and the average error between predicted and actual workload levels is 0.6. This shows that within a certain error range, the predicted results of VACVP multi-resource occupancy model are correct and can accurately reflect the actual workload levels.

In the study of workload, the relationship between predicted value and actual value is validated. In Fig. 11, The trend of the two curves is basically consistent and relatively consistent, reflecting the positive correlation between them. At the same time, the accuracy and reliability of the VACVP workload prediction model are demonstrated.

Fig. 11.
figure 11

Workload tendency comparison

Full size image

4 Discussion

By comparing the two workload values, it is found that the predicted workload value is 0, while the actual workload value is not 0 in a certain period of time, it is because we neglect that the dispatcher has been monitoring. To solve this problem we can add a level to the definition of visual part: monitoring. There is monitoring in the whole process of duty, so there is always a non-zero value. For the other hand, due to the work plan is rough and can not be accurate to every moment, its values can be discrete, so in the process of prediction, there will be a situation that the predicted value is null between the two task types. In this solution, we can refine and improve the working log files to achieve time continuity.

5 Conclusion

In this paper, a multi-resource occupancy model VACVP is used to predict the workload of dispatchers in power grid for serial and parallel tasks. At the same time, a comprehensive evaluation model based on physiological parameters is constructed to validate and compare the prediction results. As shown from the experimental results there is a certain gap between the two kinds workload value and the average error can be maintained within 0.6 level. Furthermore, the trend of the two values in the same time period is basically consistent. These results reflect the validity and accuracy of the workload forecasting model.

The forecasting model proposed in this paper is generally used to predict the load in the actual working situation of the power grid. The accuracy and continuity of this method will be further improved and verified in future experiments.