Keywords

1 Introduction

1.1 Safety Inspection Can Be Unreliable Due to Inspector’s Observation Miss

Construction has been considered as one of the most dangerous industries globally. In the U.S., there are 6-10 accidents occur on jobsite every day [1]. According to the statistic from U.S. Bureau of Labor Statistics in 2013, accidents in construction had led to 828 deaths, which ranked the first place among all industries. These accidents also resulted in great direct and indirect economic loss, which accounted for 8 % of total costs of construction projects [2]. In U.K., workers in construction only took up 5 % of the employment nationwide, but accounted for 31 % of the work-related casualties [3]. Given the severe situation of construction safety, a variety of risk-controlling methods have been taken to lower the rate of fatal accidents and minimize both casualties and economic loss.

Safety inspection, as one of them, has commonly been applied by construction companies worldwide to control risks by early detection and correction. However, since safety inspections in construction are mostly conducted by human inspectors through observation, their performance in practice may not be as reliable as generally assumed. Generally, their mistakes can be categorized into two types [4], (I) falsely judging the scenario in safety to be at risk, and (II) missing observation of potential risks during inspection. Type I, though results in a waste of resources of production and safety management, is economically manageable and harmless to workers, while type II may cause serious casualties, as well as immeasurable economic loss. Beside the severe consequence, Type II could be evitable regardless of inspector’s working experience. In fact, inspectors usually make mistakes in visual inspection due to undesirable jobsite condition (noise, lighting, etc.), various distractors along with targets, unreasonable inspection plan (frequency, sampling mode etc.), and lack of training or working experience [5, 6]. As revealed in a simulation experiment on construction inspectors [7], subjects with higher working age tended to have higher detection rate of potential risks. Nevertheless, even those with over ten years’ experience could hardly identified 80 % of all the inbuilt risks within the test, which means there were up to 20 % of risks missed by inspectors in the visual observation. Thus, mistakes of type II, missing observation of risks, are much more fatal to the reliability of safety inspection for risk controlling, and therefore require more attention from both researchers and construction companies.

1.2 Observation Miss Results from Excessive Inspection Contents

As an integration of potential risks summarized from previous accident reports and individual inspection experience, checklist is applied as a comprehensive reference for the inspectors to avoid missing observation of risks appear in construction site, and ensure an effective inspection through item-by-item examination. However, as the complexity of workspace keeps growing these days, the number of items on the checklist increase dramatically with potential risks, making it exhausting for the inspector to simultaneously map all the checklist items with potential risks in limited inspection time. For instance, in the safety inspection of amusement rides, inspectors have to go through more than fifty items on their standard checklist during their 1–2 h inspection [8]. It is the same case in construction industry. Since the construction site involves hundreds of elements, including materials, equipment and workers, it is almost out of questions for inspectors to simultaneously map all the checking items with accompanying risks [9]. To make it worse, there is ambiguity in the description of inspection contents. Sometimes it may include such expression as to ensure the working environment being ‘considered’ [10], or sometimes it may leave the inspectors to determine what is ‘proper’ based on personal knowledge [11]. The existence of such ‘grey area’ may bring extra cognitive burden to the inspector in making qualitative decisions when matching these ambiguous checking items to potential risks [12]. As a result, most inspectors tend to perform a holistic observation instead, taking the checklist as a reminder, and only have a quick review on it before or after the overall inspection. This may probably undermine the function of checklist as a reference of item-by-item inspection, leaving chance for inspectors to miss observation of potential risks.

1.3 Prioritizing Risks by Construction Stage Can Be Effective in Reducing Observation Miss

To relieve the inspectors of overload, making it possible for them to have an item-by-item examination, it is necessary to cut down the number of checking items by giving priority to some of them under certain criteria. Prioritizing the checking items (Risk-based inspection, RBI) by the level of overall impact is a common practice in this purpose. Inspection based on this criteria is called Risk-based Inspection (RBI). RBI was first applied in chemical engineering, when looking into the safety inspection on the container and piping system of oil and natural gas. It was found to be useful in locating the emergent risks, providing guidance on immediate actions for inspectors and safety managers to prevent leaking accidents [13]. Motahari [14] summarized the risks of construction process of oil drilling platform, and then used the RBI to improve the inspection procedure through the classification of risks by their priority of impact. Jagars-Cohen et al. [15] introduced RBI into the construction industry. By ranking the risks according to the level of impact, he figured out the most critical risks within a construction projects of highways in Texas, and provided practical solutions for Texas Department of Transportation (TxDOT) to tackle with the shortage of inspectors and the problem of overload by giving priority to those critical risks during inspection. However, since the risks most likely to arise at construction site may be time-varying, critical risks identified under this criteria, which remain unchanged throughout the construction process, will not always represent the risks most likely to arise at the moment of the inspection. Although Zhang and Chi [16] proposed a time-varying inspection model, how does the model influence the observation patterns and improve inspection performance (reducing observation miss) remain unclear.

This study asked how inspectors miss observations on potential risks despite the aid of comprehensive checklist and tried to come up with an effective solution. The study examined the current status of safety inspection in construction industry, including the workload of inspectors and how they use checklist under such circumstance. Then, a theoretical model from cognitive perspective was established to clarify the nature of observation miss. Finally, based on the original checklist, a dynamic checklist on elevator installation was created under the guidance of installation manual, and the effectiveness of this checklist was verified through a simulation experiment using eye tracking device. By comparing the inspection performance fixation figure of subjects in both control group (applying original checklist) and experimental group (applying the dynamic checklist), we made conclusions on whether the dynamic checklist help to reduce the inspector’s observation miss by drawing more attention of them on potential risks that most likely to arise in certain installation procedure.

2 Methodology

2.1 Causal Model of Observation Miss

Observation miss of inspectors was studied on the foundation of the theory of selective attention under load, which addresses how people’s visual attention on certain targets is affected by different levels of perceptual load and cognitive control load. Literatures on safety inspection and visual search were deep reviewed to identify direct and indirect factors leading to observation miss. The approach of information integration was then applied to form a rough picture of this model. Interviews with 6 inspectors from the company was also conducted to examine whether the theoretical connections in this model were consistent with the real situation according to their personal experience. Given the replies in these six interviews and the information summarized from the literature on load theory of selective attention, the causal model of observation miss was proposed as followed (Fig. 1).

Fig. 1.
figure 1

The causal model of observation miss

Full size image

In this model, excessive inspection contents leads to overload in working memory of the inspector to keep all of them in mind during the observation, while the ambiguous description of checking items adds to this burden, for the inspector may have a hard time judging whether an actual scenario meet the safety requirement described on the checklist. These two obstacles cause an increase in the difficulty of searching potential risks, and result in high cognitive-control load for the inspector. According to the load theory, this will then strengthens the distraction effect from common distractors existing in construction site, like scattered objects, construction noise, important phone calls and unnecessary talks. Finally, the distraction effect will be strong enough for the inspector, even an experienced one, to miss observation of underlying risks, which definitely impairs the reliability of inspection system.

2.2 Experimental Design

Before conducting the simulation experiment, a dynamic checklist oriented by the installation process of an elevator needs to be created ahead. In this study, a checklist for Fatal Prevention Audit (FPA) of an elevator company, containing 81 items of potential risks, was applied as the original version of this process-oriented checklist. 11 installation stages of elevators were also identified from an installation handbook of Generation II Comfort, which is the widest used type of elevator currently. Then, all the 81 items of risks were matched to these 11 installation stages to create a process-oriented checklist, basing on the judgment that whether a risk item was connected to the tasks of a particular stage. The connection here referred to the concurrence of elements (worker’s movements, applied objects or tools used) appeared both in the description of a certain item on the checklist and tasks under a certain stage on the handbook. Finally, the mapping result was rectified and confirmed by two experienced safety auditors from the company through face-to-face interviews.

As for the simulation experiment, this study selected 40 photographs taken at 4 typical locations (Hoistway, Pit, Machine Room, and Storeroom) from different jobsites to create a virtual environment of elevator installation. These photographs were of the same installation stage (‘Rail installation’) in contents. Then, they were numbered and organized into 4 PPT slides, a single slide for photographs taken at an individual location, to simulate the real inspection context. 43 check points were included within these photographs, containing 30 risks of three types, worker’ unsafe behavior, unsafe status of objects and unsafe working condition. These risks were determined through the selection of photographs to be as wide ranging in nature as possible, with the constraint of not being spatially conflicting to the others, to allow the inspectors to completely show their ability in detecting various types of them. However, as the storeroom was seldom the place where fatal accidents occurred, only 2 risks was set in this location. The final distribution of risks in 4 locations was presented in Table 1.

Table 1. The distribution of photographs and risks at different locations
Full size table

Five inspectors from the Beijing branch of the company were selected as subjects for this experiment, with working age ranging from 1.5 years to 25 years. Selecting subjects of wide-range working age from the same branch can avoid the interference of personal experience as well as inspection requirement of different branch on inspection performance in this experiment.

The experiment consisted of two sections, a pre-test and a pro-test. In the pre-test, subjects were provided with identical FPA checklists in original versions, and asked to examine this virtual environment through the observation of photographs in 4 slides to detect as many of hidden risks as possible. Once a risk was detected in a photograph, subjects were required to write down the number of photograph next to the relative item on the checklist. The time limit for this section was 13 min. In the pro-test, subjects were provided with a process-oriented FPA checklist specifically for the stage of ‘Rail Installation’. The process-oriented FPA checklist remains in the style of the original one, except that items related to Rail Installation were highlighted by different color. Then they were asked to do the examination again based on the detection result of pre-test, and keep records of risks further detected as what they did in the first section. The time limit for this section was 7 min. The experiment was conducted in turns, and subjects were allowed to examine these 4 location in the random order during the test according to their preference in daily inspection, simply by switching the slides through hyperlinks. After the experiment, 2 indicators were calculated to evaluate the inspection performance of each subject, illustrated as followed,

$$ {\text{Detection}}\;{\text{Rate}}\;({\text{DR}}) = \frac{{{\text{No}} .\;{\text{of}}\;{\text{risks}}\;{\text{corrected}}\;{\text{identified}}}}{{{\text{Total}}\;{\text{no}} .\;{\text{of}}\;{\text{hidden}}\;{\text{risks}}}} $$
(1)
$$ {\text{Judgment}}\;{\text{Accuracy}}\;({\text{JA}}) = \frac{{{\text{No}} .\;{\text{of}}\;{\text{risks}}\;{\text{corrected}}\;{\text{identified}}}}{{{\text{No}} .\;{\text{of}}\;{\text{detected}}}} $$
(2)

In Eq. (1), risks corrected identified referred to those detected by subjects with the correct decisions on corresponding checking items. Total number of hidden risks here was 30, as introduced above. In Eq. (2), the risks detected referred to items with photographs numbers aside. These two indicators were both calculated and statistically compared in pre-test and pro-test to determine whether the application of process-oriented checklist causes any decrease in observation miss without sacrificing the accuracy of judgment.

To monitor the shifting focus of subjects on these photographs during the inspection process and collect data on attention distribution, an eye tracking device was applied for subjects to wear all the time in this experiment. The equipment used was SMI iView X, with sampling frequency 50 Hz of eye movement. After the experiment, the image data of simulation experiments were automatically recorded in the software of BeGaze for further analysis. To better understand the distribution of subject’s attention on hidden risks, ‘Areas of Interest’ (AOI) were defined to locate the exact region of risks appeared in specific photographs, based on recorded images of inspection process. With the help of BeGaze, we could have the value of several indexes for eye movement in defined areas on the screen. Given these indexes, we could have the value of an indicator addressing search efficiency, calculated as followed,

$$ {\text{Search}}\;{\text{Efficiency}}\;({\text{SE}}) = \frac{{\sum\nolimits_{i = 1}^{n} {(AF)_{i} \times (FC)_{i} } }}{Total\;Time\;of\;inspection} $$
(3)

In Eq. (3), \( AF_{i} \) referred to average fixation time on a certain AOI, while \( FC_{i} \) referred to fixation counts. As a result, the product of this two indicators would be the total time of fixation spent on a potential risks, which was taken as the valid time in one’s searching task during the safety inspection. Besides, the total time of inspection was not 13 min in the pre-test, or 7 min in the pro-test. Instead, the time of reviewing checklist during the inspection should be deducted, for the subjects then were not actually conducting the observation of photographs. The deduction in total time could be clearly identified and calculated from the image data. This indicator was calculated in both pre-test and pro-test with a statistical comparison to determine whether the application of process-oriented checklist causes any decrease in the proportion of inspector’s observation time spent on distractors rather than hidden risks.

3 Findings

3.1 Inspection Performance on Risks Detection

Table 2 below provides the average detection rate in both pre-test and pro-test. According to the results of t-test, there is a significant increase in detection rate (sig. = 0.01/0.03 < 0.05) after applying the process-oriented checklist in the pro-test, from 15 % to 24 % (23 %), whether the items unrelated to rail installation, yet being detected in pro-test, are included or not. Table 3 provides the average judgment accuracy in both pre-test and pro-test. The result of t-test reveals that there is no significant change in judgment accuracy (sig. = 0.93/0.92 > 0.05) after applying the proposed checklist in the pro-test. Figure 2 below takes into account the number of risks corrected identified versus the working age of inspectors. As is shown, R2 values are very low (R2 = 0.0103, 0.0727, 0.1314 ≪ 1), being far away from 1, in both the pre-test and pro-test, indicating no direct relationships between the detection rate of inspectors and their working experience. It’s the same story of judgment rate (R2 = 0.0081, 0.0128, 0.0138 ≪ 1), indicating no direct relationships between the judgment rate of inspectors and working experience (Fig. 3).

Table 2. The t-test for detection rate
Full size table
Table 3. The t-test for judgment accuracy
Full size table
Fig. 2.
figure 2

The relationship between detection rate and working experience

Full size image
Fig. 3.
figure 3

The relationship between judgment accuracy and working experience

Full size image

3.2 Inspection Performance on Searching Efficiency

Table 4 below shows the percentage of valid search for 5 subjects in both the pre-test and the pro-test. To be mentioned, the data of subject No.1 is abandoned because of too many head movement during the experiment, making the definition of AOI (Area of interest) to be inaccurate. The other missing data in this table are all from the pro-test. The reason is that since all the photographs are examined in the pre-test, subjects may not have a double check on each one of them in the pro-test, given the limited inspection time in this section. As for the data left, there is a significant increase in the proportion of valid search (Table 5, sig. = 0.03 < 0.05) after applying the process-oriented checklist in the pro-test, indicating that search efficiency of inspectors has been improved by the proposed checklist.

Table 4. Search efficiency in two sections
Full size table
Table 5. The t-test for search efficiency
Full size table

4 Discussions

As illustrated in the causal model, the reason why inspectors miss observation of potential risks despite the aid of checklist was that the usage of checklist usually deviates from its original purpose because of heavy cognitive-control load. In practice, the inspectors seldom used it simultaneously while examine the construction site, for the checklist are too complicated in contents without reasonable structure and indication of priority. Instead, checklist was commonly used as a reminder of the inspection contents before or after a holistic observation. These findings suggested that inspectors actually need a well-structured checklist corresponding to the nature of technical system being examined. In response to the defects, a dynamic checklist oriented by installation stage was proposed in this research. Then the experiment results provided a sustainable support to the effectiveness of proposed checklist in reducing observation miss by presenting an increase in both detection rate and search efficiency. To be mentioned, the detection rate may only serve as a reference for the improvement to avoid missing observations of risks. Because the risks detected in the pro-test may include those already been detected in the pre-test. Thus, the increase in searching efficiency is a more substantial support for the effectiveness of process-oriented checklist, as it revealed that more attention had been focused on potential risks after applying the proposed checklist. However, due to the difficulty of accessing qualified subjects, the sample size is not large enough to reach a conclusion of statistical significance. Besides, despite the effectiveness of structured checklist, it is better to be used as a complement after holistic observation in practice. Because there will always be defects in the design of a checklist, and a structured checklist with fewer inspection contents at a certain time may have risks in missing observation of targets outside the checklist.

5 Conclusions

The research illustrates the nature of observation miss in safety inspection through a causal model. The complexity of checklist, including having too much checking items as well as being ambiguous in descriptions, will increase the cognitive-control load for the inspectors. In that case, inspectors will then shift their inspection mode of inspectors from item-by-item examination to holistic inspection. As the high cognitive-control load strengthens the effect of interference from existing distractor on jobsite, inspectors will finally be prone to missing observations of potential risks during their holistic inspection. These findings provide a convincing answer to the question that why missing observation of potential risks still happens in safety inspection of construction industry given the comprehensive checklist applied by the inspectors. Besides, the role of working experience of the inspector was also clarified to have little influence on his inspection performance. Future research should aim to carry out a similar experiment on validating the dynamic checklist with larger sample. Although augmenting cognitive load of inspector is found to be effective in reducing observation miss, future research should focus on how to dynamically propose inspection items according to the location of inspectors and construction procedures.