A Multidimensional Workload Assessment Method for Power Grid Dispatcher

Abstract

Dispatcher’s error is an important factor affecting the safe operation of power system. One of the main causes of human error is inappropriate workload. Due to the particularity of the power dispatching work process, existing workload measures are not ideal for power dispatcher. According to the human information processing model, combined with the actual work of dispatchers, this article proposed a novel method for dispatcher workload assessment. It considered dispatcher’s workload from four dimensions: information perception, speech output, action output and attention. Video, audio and physiological monitor were deployed to acquire descriptive features. The frequency of incoming calls was extracted to describe information perception. Short-term energy and spectral entropy of the speech signal were extracted to describe speech output. Body movement speed was extracted to describe action output and heart rate was used to describe attention. The method was applied to an experiment in the dispatcher training simulator involving qualified power dispatchers. The experimental results showed that the proposed method was applicable and it can effectively reflect changes of dispatcher’s workload during troubleshooting tasks.

1 Introduction

With the progress of science and technology, the reliability of automations in complex systems has been greatly improved. It has been accepted in several areas (e.g. civil flight, air traffic control, nuclear plants and road traffic, etc.) that human factors have gradually become the primary threats to safety. Above 70% of accidents are relevant to human errors [1].

A power dispatching room is a typical human-in-the-loop complex system. In the power dispatching room, dispatchers handle with customers’ requirements and release commands to power plants via communication system so as to make sure the energy supply meets customers’ demands. Meanwhile they also need to monitor the status of the power grid so that the energy won’t damage the plants and systems. As the dispatchers do not directly operate the electricity production equipment, their cognition and decision on the grid status play a key role in safe operation of the power system [2].

The relationship between safety risk and workload is like a u-shape curve [3]. Too low or too high of the workload will both increase the risk to the system. When workload is too low, it is insufficient to maintain operator’s situation awareness, and would decrease the speed and accuracy of operator’s reaction. That would be dangerous especially in emergency situations. When the workload is too high, it might exceed operator’s capability and would also degrade the quality of their performance.

Workload is an abstract concept which cannot be measured directly. According to previous studies, various techniques have been proposed to reflect workload. They could be classified into three broad categories [4]: (1) Subjective ratings, such as NASA-TLX, SWAT, Bedford and etc. these methods tend to quantitative operator’s experienced workload through a set of elaborately designed questionnaires. (2) Performance measures, such as primary task performance and secondary task performance. These methods are based on the supply-demand relationship of mental resource. (3) Physiological measures, such as ECG, EOG, EEG and etc. these measures are based on the adjustment mechanism of autonomic nervous system.

However, the special work environment of power dispatcher restricts the application of the existing workload evaluation method. Firstly, power dispatcher’s work is a continuous operation with long duration. Each shift has to work 8 h continuously (3 shifts in 24 h). Therefore, continuous monitoring of dispatcher’s workload is essential for promptly detection of potential risk. Subjective rating techniques are post evaluation methods. Evaluation result can only be given after the work or every once in a while during the work. This would lead to low temporal resolution and would interfere with dispatcher’s work if it was carried out during the work. Secondly, the role of power dispatcher is more like a commander than an operator. They release orders to other departments in the grid system and the order would be achieved by field personnel. The outcome of dispatcher’s decision would return after an uncertain delay. Therefore, it is hard to use performance measures to evaluate dispatcher’s present workload. Thirdly, the main function of power dispatcher is monitoring and decision making. The mental workload is much higher than physical workload. It should be verified that whether the physiological parameters approved in other conditions are still effective in the power dispatching room.

Currently, most of the conventional power dispatcher workload assessment methods are still based on time line analysis i.e. ratio of task occupied time to available time. For example, dispatcher’s workload could be calculated as:

$$ workload = (n*t + T_{s} )/(N*t + T_{s} ) $$

(1)

where n is the total items in all the operation orders released by dispatcher, t is the average time to release one order item, T _s is the time occupied by secondary tasks, N is the maximum number of items the dispatcher can release (releasing operation orders to the control room operators is the primary task of a power system dispatcher, other tasks such as filling in the dispatch log, creating the order draft, verifying the operation order, approving the maintenance application, etc. are all secondary tasks).

Although task amount or time occupancy seem to reflect dispatcher’s work intensity, it is not an effective assessment of dispatcher’s workload. Sometimes, the operator apparently has many operation activities, however, these activities might not take much mental resources. In fact, dispatcher’s workload depends on the occupancy of his mental resource.

In order to develop an applicable method which can provide continuous and effective assessment of power dispatcher’s workload, this study proposed a novel workload assessment model which both considering the characteristics of power dispatching work and the cognitive theory. According to this conceptual model, data acquisition, data processing and feature extraction methods were also introduced in this paper. Furthermore, we conduct an experiment in the power dispatching room so as to apply the proposed methods and test its effectiveness.

2 Method

2.1 Conceptual Framework

According to the human information processing model [5], there are several stages when human performing a task: (1) Short term sensory store (STSS). During which stage human acquire the outside information. (2) Perception. During which stage a person make a quick understanding of the information’s apparent meaning. Combining working memory and long-term memory, human can make a further understanding of the situation. (3) Response selection. During which stage human weighs the pros and cons to make a decision. (4) Response execution. During which stage the brain control muscles to carry out the decision.

Besides the above four stages, there are two essential elements in the information processing model: feedback and attention. When the execution of response changes the environment and brings new pattern of information, feedback should be considered. Attention is the inherit capacity of human. It represents the occupancy of mental resource.

Note that there are not explicit boundaries between some stages in the information processing model. Those stages are carrying out swiftly in mind (e.g. STSS, perception and response selection, etc.) and cannot be measured separately. Furthermore, consider the separation of dispatching room and operation site and the delay of the order outcome, we do not consider the changes of dispatcher’s response to the environment for the time. Therefore, we simplify the information processing model as Fig. 1. The simplified model includes two stages (cognition stage and response stage) and the attention elements.

Fig. 1.

A simplified human information processing model.

Combine the information processing model with power dispatcher’s work. In the cognition stage, information primarily come from incoming calls. (Of course there are also visual information. However, we have not found an appropriate way to measure the amount of visual information acquired by a dispatcher). In the response stage, on one hand dispatchers respond in the form of action and movement such as keyboard input, checking for material, reaching for the phone and etc.; on the other hand, dispatchers respond in the form of speech such as giving operation orders or asking for situation. Both the cognition stage and the response stage cause dispatcher’s attention. The attention or mental resource occupancy cannot be measured directly. In this study, we use physiological reactions to reflect dispatcher’s attention.

Therefore, we build the following dispatcher workload model (Fig. 2). Workload can be reflected from four dimensions: information perception, action output, speech output and attention. Each dimension should be continuous measured. For example, information perception and speech output can be measured with audio acquisition. action can be monitored by video surveillance. Attention can be measured by physiological measures.

Fig. 2.

The conceptual model of power dispatcher’s workload.

In order to provide quantitative workload assessment result, each dimensions need to be represented by quantitative features. The following part of this section introduce our data processing and feature extraction methods.

2.2 Feature Extraction

Information Perception.

In information perception dimension, we use “the frequency of incoming calls” to reflect task demand imposed on power dispatcher during the cognition stage.

In order to obtain the frequency of incoming calls, some audio processing technologies are required. In consideration of the frequency selection characteristic of band pass filter i.e. it can pass frequencies within a certain range and reject frequencies outside that range, we decide to use the Type I Chebyshev filter to separate ringtone signal from audio data [6]. The Amplitude-frequency relationship of a type I Chebyshev filter is as follows:

$$ \left| {H_{n} (j\omega )} \right| = \frac{ 1}{{\sqrt { 1 { + } \epsilon^{ 2} T_{n}^{ 2} (\frac{\omega }{{\omega_{ 0} }} )} }} $$

(2)

where $\epsilon$ is the ripple factor, ω₀ is the cutoff frequency and T_n is a Chebyshev polynomial of the n^th order. The passband exhibits equiripple behavior, with the ripple determined by the ripple factor $\epsilon$.

In order to design a desired band pass filter, the frequency characteristics of the ringtone signal in real dispatching room should be studied. In addition, the lower passband frequency, the higher passband frequency, the lower stopband frequency, the higher stopband frequency, the passband ripple and the stopband ripple should be set up carefully.

After ringtone signal has been separated from the audio data, the frequency of incoming calls can be easily obtained by counting the number of ringtones in particular time intervals.

Speech Output.

In the speech output dimension, we use “short-term average energy” [7] and “spectral entropy” [8] to reflect power dispatcher’s work intensity during voice communication.

First of all, a band pass filter is also necessary in this dimension. By analyzing the frequency characteristics of real dispatchers’ voice, a band pass filter should be designed to separate voice signal from the audio data. Meanwhile, other sound such as ringtones and noises should be inhibited.

When voice signal is extracted from the original audio data, short-term average energy of the voice signal can be calculated by equation below. This feature describes dispatcher’s volume changes over time.

$$ E_{n} { = }\sum\nolimits_{{m{ = 0}}}^{{N{ - 1}}} {x_{n}^{ 2} (m )} $$

(3)

where E_n represents the short-term average energy of the nth frame, x _n is the nth frame of the voice signal segmented by a short-term window, N is the length of the short-term window.

Entropy is the measurement of uncertainty. Generally speaking, the more uncertain a signal, the more information it contains. In the voice signal processing area, spectral entropy can be used to describe the complexity of the speech. Spectral entropy can be calculated by the following steps:

• Segment the original voice signal x(n) into frames

• For the nth frame x _n (m), perform FFT to obtain X(f)

• The power spectral density is computed by PSD = |X(f)|²

• Normalize the PSD so that it can be viewed as a probability density function.

  $$ p_{i} (f )= \frac{{PSD_{i} (f )}}{{\mathop \sum \nolimits_{{i{ = 0}}}^{N / 2} PSD_{i} (f )}} $$

  (4)

  where N is the length of FFT.

• Finally, spectral entropy can be calculated by:

  $$ H_{i} = - \sum\nolimits_{{k{ = 0}}}^{N / 2} {p_{i} (k ) {\text{log}}p_{i} (k )} $$

  (5)

Action Output.

In the action output dimension, “body movement speed” is used to describe dispatcher’s work intensity.

Image processing techniques are applied to compute body movement speed. The processing procedure is illustrated in Fig. 3. There are two major tasks in this procedure: motion detection [9] and skin detection [10].

Fig. 3.

The image processing procedure to obtain dispatcher’s body movement speed.

Motion detection takes 4 steps. (1) color image is converted to grayscale image. (2) “Opening Operation” is performed to reduce noise and filling holes in the image. (3) inter-frame difference is calculated so as to detect the moving part in the scene.

$$ D_{n} \left( {x,y} \right) = \left| {f_{n} \left( {x,y} \right) - f_{n-1} \left( {x,y} \right)} \right| $$

(6)

where $f_{n} \left( {x,y} \right)$ is n^th frame of the video, $f_{n - 1} (x,y)$ is the (n−1)^th frame of the video. $D_{n} \left( {x,y} \right)$ is the difference of adjacent video frames. (4) By using threshold processing, the moving part can be explicitly distinguished from the background.

$$ M_{n} \left( {x,y} \right){ = }\left\{ {\begin{array}{*{20}c} {\text{255},} & { Dn\left( {x,y} \right) > T} \\ {\text{0}, } & {else} \\ \end{array} } \right. $$

(7)

where T is threshold. $M_{n} \left( {x,y} \right)$ is a black and white image in which moving part is white (255) and stationary background in black (0).

Skin detection takes 3 steps. (1) image is convert from RGB to HSV color space. (2) by using skin color threshold to extract body from image. This can roughly set the skin areas to white and other areas to black. (3) using region growing to obtain relatively integral block to represent body part such as face, hand and etc.

Take the intersection of motion detection result and body detection result. This would provide a relatively robust representation of moving body. By calculating the area of all the white blocks, the result could represent dispatcher’s body movement speed.

Attention.

In the attention dimension, we use heart rate to represent the occupancy of mental resource. According to previous studies, heart rate has been proved to be an effective indicator of mental workload in various fields [11]. When operator encounters with more challenger tasks or feels more stressful, his/her heart rate would usually increase significantly.

Moreover, heart rate is easy to measure. It does not have to be measured by bulky laboratory instrument. Nowadays, heart rate can be measured unobtrusively by tiny remote sensors. For example, the Photoplethysmography (PPG) technology detects heart pulse by sensing the slight changes in the color of skin caused by blood flow. The PPG sensor is often very small which can be integrated into a common watch or wristband. The most important is that the PPG technique can provide satisfactory measurement accuracy [12].

Integrated Assessment.

The proposed method requires different types of measure data and they may come from different instruments. In order to associate all the data for integrated workload assessment, time synchronization is an inevitable problem. In this study, we use sample timestamp to synchronize all the data (the premise is that all devices’ internal clocks have been synchronized in advance).

After all the features have been synchronized, principal component analysis is used to examine the correlation between the features and combined them by using the following equation [13]:

$$ W{ = }\sum\nolimits_{{i{ = 1}}}^{n} {\beta_{i} z_{i} } $$

(8)

where, W is the overall dispatcher workload assessment index. n is the total number of principal components. β_i is the percentage of variance explained by the i^th component. z_i is the component score of the i^th component.

3 Experiment

3.1 Participants and Apparatus

Ten qualified male power dispatchers participated in this experiment. Their age is between 28–32 years old and with the working experience of 3–6 years. The experiment was carried out in the Dispatcher Training Simulator (DTS) which is a system that can simulate the behavior of electrical network under various conditions. In this experiment, the instructor set up a series of adverse scenarios for them including different kinds of equipment malfunctions. The participants did not know these setting in advance. They were supposed to quickly analyze the situation and give out reasonable solutions.

During the experiment, video data was collected by a HD wide-angle camera (Fig. 4). There is a microphone integrated in this camera, so that audio data can be collected concurrently. The video and audio data are stored offline in the camera’s build-in memory card. We continuously collected dispatcher’s heart rate with a heart rate watch (Mio Alpha; Physical Enterprises Inc., Vancouver, BC) which use PPG technology. Therefore, it was unnecessary for participants to wear sensors such as chest strap near their heart. Heart rate data is transmitted in real time to a receiving device via Bluetooth. In this study, the receiving device is a laptop with Bluetooth module, and we have developed a special software that can receive and store the heart rate data with sampling time-stamps.

Fig. 4.

Experimental setting in the DTS. (a) a HD wide-range camera was install above the monitor. (b) the dispatcher is wearing a heart rate watch on his left wrist

3.2 Experiment Procedure

The experiment was conducted in accordance with the following steps: (1) Before experiment started, experimenter installed the camera to ensure that it did not interfere with dispatcher’s sight and fully cover the dispatcher’s working area. (2) Experimenter synchronize the internal clocks of the camera and the heart rate receiving laptop. (3) After participant’s entering to the DTS, experiment helped him to wear the heart rate watch, started the heart rate collection function and test the validity of the data. (4) Experimenter started camera’s recording function and the data receiving software in the laptop. (5) Experimenter left the DTS and started the experiment. Each trial lasted about one and a half hours. (6) After the experiment, the experimenter terminated the collection and exported the data.

3.3 Result

Information Perception.

The waveform of a segment of ringtone and its FFT are illustrated in Fig. 5(a) and (b). According to the frequency characteristics of the ringtone, we design a band-pass filter. The amplitude-frequency curve of the band pass filter is shown in Fig. 5(c). Specifically, the lower passband frequency was 3300 Hz, the higher passband frequency was 3450 Hz, the lower stopband frequency was 3200 Hz, the higher stopband frequency was 3500 Hz, the passband ripple was 1 dB and the stopband ripple was 5 dB.

Fig. 5.

Design of a band pass filter for ringtone extraction. (a) the waveform of the ringtone signal. (b) FFT of the ringtone signal, (c) The amplitude-frequency relationship of the band pass filter which is used to extract ringtone from audio signal.

A segment of audio signal containing both ringtone and speech was shown in Fig. 6(a). After band-pass filtering, the result was shown in Fig. 6(b). As can be seen, this band pass filter can effectively separate ringtones from original audio.

Fig. 6.

Result of band pass filtering. (a) the waveform of a segment of audio signal containing both speech and ringtone. (b) Signal after band pass filtering.

Speech Output.

Another band pass filter was designed to separate speech from audio. Its amplitude-frequency curve was illustrated in Fig. 7. Specifically, the lower passband frequency was 100 Hz, the higher passband frequency was 800 Hz, the lower stopband frequency was 50 Hz, the higher stopband frequency was 850 Hz, the passband ripple was 1 dB and the stopband ripple was 5 dB. As can be seen that after band pass filtering, ringtones in the original signal has been weaken and the speech became more significant.

Fig. 7.

The amplitude-frequency curve of the band pass filter which is used to extract speech from audio signal.

After band pass filtering, the short-term energy and spectral entropy of the speech signal were calculated and plotted as illustrated in Fig. 8(a) and (b) respectively.

Fig. 8.

Short-term energy and spectral entropy of a segment of speech signal. (a) Short-term energy. (b) Spectral entropy.

Action Output.

By associating the results of skin detection and motion detection, the moving body part was detected as illustrated in Fig. 9.

Fig. 9.

Moving body has been extracted from video image.

The body movement speed can be calculated simply by counting the area of the white blocks in the result image. The changes of body movement speed over a period to time was illustrated in Fig. 10.

Fig. 10.

The changes of body movement speed over time.

Attention.

The original heart rate data was illustrated in Fig. 11(a). Due to body movements, sometimes the heart rate sensor would be poorly contacted. This would result in some outliers. After a simple threshold processing, we got the ideal heart rate data, as illustrated in Fig. 11(b).

Fig. 11.

The result of threshold processing of the heart rate data. (a) Original heart rate data. (b) Filtered heart rate.

Integrated Assessment.

By performing PCA, five principal components were extracted from the features. The Eigen value and percentage of variance explained by each component was illustrated in Table 1.

Table 1. The result of principal component analysis.

According to Eq. 8, the overall workload of a power dispatcher can be calculated:

$$ W = 0. 6 9 0 2z_{ 1} + 0. 1 6 3 7z_{ 2} + 0. 1 3 7 9z_{ 3} + 0. 0 0 4 2z_{ 4} + 0. 0 0 4 0z_{ 5} $$

(9)

Figure 12 shows the changes of dispatcher’s overall workload over time.

Fig. 12.

The result of dispatcher’s overall workload.

4 Discussion

As can be seen from Fig. 12, the overall workload waveform significantly rises at 100000 ms and 150000 ms. By examining the original video and audio data, it was found that at those two moments there happened to be two incoming calls respectively. One is from dispatcher’s superior asking for the condition of the malfunction, and the other is from field staff at the local transformer substation consulting troubleshooting proposal. For those two calls, dispatchers could not give definite replies because the cause of the malfunction had not been revealed yet. During these periods, the dispatcher needs to consider various factors to comprehensively evaluate the current situation and make appropriate judgement. It requires the brain to conduct more analysis and processing than usual. As a result, the resource occupancy should be higher. To a certain degree, this prove the validity of the proposed method.

There are still some limitations in this study. Firstly, in the information perception dimension, we did not consider information. This was because that we have not found an effective way to measure the amount of visual information perceived by dispatcher. Although eye tracking was a potential way [14], however, based on our actual inspection, we found that almost all the dispatchers in this study wear glasses. This limit the application of the existing glasses-type eye tracker. Considering the wide distribution of dispatcher’s visual attention (they often need to monitor 5 to 6 screens at the same time), desktop eye tracker was also not applicable. Secondly, due to limited time, this study did not go further into the meaning of each principal component. Actually, it was very important for us to understand the mechanism of human factor risk in power dispatching. Thirdly, this study was based on data sample from relatively few trials. This is far from enough to reflect the general situation in the dispatching work. In future study, more experiments need to be carried out in both DTS and real dispatching room. Statistical analysis should be done to test the validity of the proposed method.

5 Conclusion

In this paper, we analyzed the human information processing model, and combined with the actual work of power grid dispatcher, put forward a novel dispatcher’s workload evaluation method. The method considered that dispatcher’s workload can be describe from four dimensions: information perception, speech output, action output and attention. In each dimension, the feature extraction method had been introduced in detail. Principal component analysis was used to combine all the features into an overall workload assessment. The proposed method was applied to an experiment involving certified power dispatchers. The experimental results proved the validity of this method to some extent. In future research, the sample size need to be increased. We will test the validity and reliability of the proposed method more rigorously with statistical analysis.