Insulator Segmentation Based on Community Detection and Hybrid Feature

Abstract

Image segmentation is an important prerequisite for automatic detection of insulator’s surface defects. It is difficult to remove the interference by using traditional methods to segment insulator from aerial images with wires adhering to insulators, pole tower in a large proportion and complex background. To solve these problems, this paper proposes a segmentation method with complex network community detection and hybrid feature for aerial insulator image segmentation. We implement community segmentation for achieving a higher accuracy by using the similarity between pixels. In this method, the image is segmented into super-pixels, then the features of color and texture are calculated to get hybrid feature. Next, we set up a super-pixel network by calculating Gauss similarity of hybrid feature information. Finally, we use the complex network community detection method to extract the insulator. The experiment results demonstrate the presented method is robust, efficient and accurate.

This paper is supported by the project “Research on Deep Detection and Defect Recognition Technology for Electric Power Equipment Facing Transmission Line Patrol Inspection of Unmanned Aerial Vehicle” under grant number AI83-19-006 in China Electric Power Research Institute and “Research and Application of Operation Robot System in Electric Power Industry” under grant number 2018YFB1307400 in National Key R&D Program of China and the National Natural Science Foundation of China (NSFC) under grant number 61871182, 61401154, 61773160, 61302163, by Beijing Natural Science Foundation under grant number 4192055, by the Natural Science Foundation of Hebei Province of China under grant number F2016502101, F2017502016, F2015502062, by the Fundamental Research Funds for the Central Universities under grant number 2018MS095, 2018MS094, and by the Open Project Program of the National Laboratory of Pattern Recognition (NLPR) under grant number 201900051.

1 Introduction

Long-term variable weather and mechanical fatigue affect the state of insulators on the transmission lines. As a result, insulators would be cracked or defected on their surface, which affects the safety of transmission line [1]. Manual detection not only wastes time, but also is dangerous. Currently, aerial patrol, as an efficient way of transmission lines monitoring, has been put into a practical application. Extracting the insulator targets accurately from images is the essential premise for detection of surface defects by analyzing large size aerial video images.

Threshold segmentation methods can extract the target based on its gray component [2]. However, inordinate amount of edge details is lost. For images with complex background, they can hardly do anything to segment target. Ma and An [3] adopt region location method to extract insulator targets from images. Combining OTSU method, it locates blue insulators by using images’ chrominance and saturation. It is the problem that the method just considers gray feature and ignores spatial features of pixels. It’s hard to get ideal segmentation results of aerial images, which are complex, noisy and low resolution. Iruansi [4] and Wu [5] all use active contour model for insulator segmentation. Although the results of their methods are better than others, manual selection of initial contour would increase operations of user interaction. Their method is only suitable for images with simple background and large strings of insulators. Moreover, their time complexity is high due to the multiple iterations. Literature [6] puts forward a kind of composite insulator images segmentation method based on improved color differences. An image segmentation method for insulator based on ant colony classification and weighted variable fuzzy c-means (FCM) is proposed in this study to overcome the limitations of the traditional method for insulator in transmission line [7].

The above methods get a better result for images with simple background and grayscale difference between insulators and other targets. An additional worry is that minor local changes in the complex image target (such as, partial occlusion) maybe lead to image segmentation or edge extraction failure. The shortcomings contribute to low efficiency and weak robustness of segmentation. In addition, their level of automation needs to be improved. Complex network community detection is based on similarity of every node and link strength of every edge to partition image network. In the paper of Girvan and Newman [8], they proposed an algorithm about complex network community structure detection, namely famous GN algorithm.

Literature [9] combines community detection with minimum spanning trees to implement image segmentation processing. However the method only segments some simple background images and can’t segment images with target adhesion. Amiri and Abin [10] select single feature as the input of community detection. However the image can only be segmented into some monochrome targets by using the method. It can’t extract target as an entirety and different parts of the target are different in color.

Community detection method is sensitive to the entirety of target. It uses community to express target. So it should to be possible to extract insulator as an entire community from aerial image with target adhesion for the uniqueness and integrity of insulators. To segment the images with complex background, we use hybrid feature as the input of community detection, and it combines the color feature and texture feature by setting an appropriate weight. The hybrid feature makes community detection appropriate for most aerial images.

In order to solve the problems about extracting insulator target from aerial images with wires adhering to insulators, a large proportion of pole tower and complex background, we introduce the complex network community detection into the insulator image segmentation and propose a segmentation method for aerial insulator image based on community detection and hybrid feature. The method not only removes false targets that are similar to insulator effectively, but also improves segmentation efficiency.

The rest of the paper is organized as follows. Section 2 describes the specific implementation processes and procedures of the proposed method; Sect. 3 mainly shows results of our experiments and their analysis and comparison; Sect. 4 is conclusion of the proposed method.

2 Segmentation Based on Complex Networks Community Detection

In this paper, we propose an insulator segmentation method based on community detection and hybrid feature. By observing a large number of aerial images, there are many significant characteristics, such as complex background, similar feature, more false targets and so on. Overview of this method is shown in Fig. 1.

Fig. 1.

Overview of the aerial insulator segmentation method based on complex network community and hybrid feature.

2.1 Super-Pixel Segmentation

Complex network community detection has two problems, which are low accuracy of community segmentation and time-consuming. If we consider pixels as nodes to detect communities, the accuracy of the results will be affected, and the amount of calculation will be greatly increased. At present, super-pixel segmentation [11] is a method of image pre-processing, which can keep more information of image and reduce the computational burden of image post-processing. There are many adjacent pixels with similar features of targets and background in aerial insulator image. It clusters similar pixels together to form some large pixels that will be regarded as nodes of image network, which improves accuracy of community detection segmentation and reduces time complexity.

The super-pixel method used in this paper is Turbopixel (TP) [12]. It can directly control the number of super-pixel and has a fast computation speed. Segmentation result of super-pixel is shown in Fig. 2.

Fig. 2.

Super-pixel segmentation result. (a) The aerial image; (b) Result of super-pixel for (a).

2.2 Hybrid Feature Generation and Networks Creation

At present, most aerial insulator images are still processed with single feature. If using separate feature entirely, we can’t remove wires, pole towers (that have similar texture features to insulator), which affect segmentation accuracy greatly.

Insulator images are complex and have many false targets. As shown in the right picture of Fig. 3, every super-pixel has regular shape and contains many characteristics, such as color, texture and so on. When the image is segmented only by using color feature or texture feature, we can’t get ideal results that can be accepted. To solve the problem, we select hybrid feature combining color feature and texture feature with a certain weight value ratio. Segmenting image by using the hybrid feature can improve the quality of results. However, different images have different weight value to get ideal segmentation results. Through making a lot of experiments, we come to a conclusion that texture feature is more important than color feature in aerial insulator images. And we use different weight values to set experiments. In the aerial insulator experiments, using 2:5 weight value ratios is better than others between color feature and texture feature to get hybrid feature.

Fig. 3.

Distribution of super-pixel segmentation. (a) Result of super-pixel; (b) Image patch distribution. (Color figure online)

According to the hybrid feature, we consider every super-pixel as node to calculate the Gauss similarity by Eq. (1).

$$ S_{ij} = \left\{ {\begin{array}{*{20}c} {\text{exp}\left( { - \frac{1}{{2\sigma^{2} }}\left\| {x_{i} - x_{j} } \right\|^{2} } \right)} \\ {0,} \\ \end{array} } \right.,\begin{array}{*{20}c} {i \ne j} \\ {i = \, j} \\ \end{array} $$

(1)

Where, S_ij is an element of the network’ adjacency matrix that means the similarity between node i and node j.

The calculated results are be defined a path between nodes. So we use the paths and nodes to create the image network (Fig. 4). Every part of the complex network has its own characteristics. For example, the links of target region are concentrated in the network. We can extract it separately by the characteristics of insulator.

Fig. 4.

Image network map.

As shown in Fig. 5, the surrounding part of the green line is the extracted insulator part. We make an experiment to set different kernel parameter of Gauss similarity. The better kernel parameter is 0.5, namely σ = 0.5.

Fig. 5.

The results of different kernel parameter of Gauss similarity. (a) is σ = 0.5; (b) is σ = 1; (c) is σ = 1.5. (Color figure online)

2.3 Complex Networks Community Detection

Complex networks community detection is based on modularity [13]. The method uses Eq. (2) to calculate modularity Q between features of each node in the network.

$$ Q = \sum\limits_{i} {\left( {e_{ii} - a_{i}^{2} } \right)} $$

(2)

The value of Q is the standard to judge the quality of community. Greater value of Q can get better community. In Eq. (2), e_ij is the value of edge between community i and community j, a_i is the value of edge between nodes in one community i.

The value of Q only judge quality of separate community. According to the fast algorithm proposed by Jordi and Alex [14], we use Eq. (3) to calculate the value of relative modularity ∆Q and cluster communities with the maximum ∆Q to get a new community. Because the greater value of ∆Q is, the more similar the two communities will be. Repeat the process until there is only one big community finally.

$$ \Delta Q = e_{ij} + e_{ji} - 2a_{i} a_{j} = 2\left( {e_{ij} - a_{i} a_{j} } \right) $$

(3)

Where, e_ij is the value of edge between community i and community j, and a_i is the value of edge between nodes in one community.

As shown in the right picture of Fig. 6, after n times’ iteration, the image network will be represented as an entirety. Each iteration will generate a new community, and the new community will be used for the next iteration. Because the similarities of every two communities are different, the sizes of final communities are different. In the picture, we can find 7 bigger communities obviously, which include insulator community.

Fig. 6.

Super-pixel segmentation. (a) Super-pixel segmentation result of 7 times iteration; (b) n times’ iteration of network.

In every community, internal nodes are linked closely in each separated small community, but external nodes are linked sparsely.

As shown in Fig. 7, all of the insulators’ super-pixels are expressed in an entire separated community, which don’t contain the wires that adhering to insulator. We still need to do more experiments to find ideal parameters to remove the small inference in the aerial image.

Fig. 7.

The result of insulator segmentation.

3 Experimental Results Analysis

In order to verify the efficiency and robustness of the method, we design three groups of experiment images, which are with simple background (Sect. 3.1), complex background (Sect. 3.2) and with different noise (Sect. 3.3). The images of experiment 1 are downloaded from Internet, the images of experiment 2 are aerial insulator images and the images of experiment 3 are collected from the aerial video of transmission line. The aerial insulator images contain complex scenes such as pole tower, insulator adhesion, a number of wires, and ground scene whose color feature is similar to insulator. The accuracy of segmentation results is evaluated by the value of intersection-over-union (IOU) and comprehensive quality [15]. IOU is defined as the ratio between intersection of segmentation result and groundtruth and their union. (Groundtruth is gotten by manual segmentation.) The greater value of IOU is, the better results would be.

$$ IOU = \frac{segment \cap groudtruth}{segment \cup groudtruth} $$

(4)

Where, segment is the result of experiment and groundtruth is the standard graph.

The comprehensive quality Y of segmentation results is made up by Probabilistic Rand Index (PRI) [16,17,18] between segmentation result and groundtruth, Variation of Information (VoI) [19] and Global Consistency Error (GCE) [20]. Its formula is as following:

$$ Y = \frac{1}{3}*\left( {PRI + \frac{1}{VoI} + GCE} \right) $$

(5)

As shown in Eq. (5), smaller VoI means better results and greater Y can get better performance.

The parameters of the experiment: the number of super-pixel is 200, the kernel parameter of Gauss similarity is 0.5 and the hybrid feature weight ratios is 2:5.

The presented method is compared with the 5 methods:

• 1. OTSU;

• 2. Segmentation based on color feature;

• 3. Segmentation based on texture feature;

• 4. Method about extracting insulator with active contour model [7];

• 5. Segmentation based on community detection and minimum spanning trees [9];

Method 4 uses a new active contour model to extract in homogeneous insulators from aerial images for overcoming the difficulties caused by texture inhomogeneity. Method 5 is community detection method by using two rounds of minimum spanning trees.

According to the thought of the literatures [7, 9] respectively, we implement method 4 and method 5.

3.1 Segmentation Experiments About Insulators with Different Material and Shape

As shown in Fig. 8(a), experimental samples are insulators with different material and shape. The segmentation results of the 6 methods are shown in Fig. 8(b) to (g).

Fig. 8.

The results of 6 segmentation methods. (a) Original image; (b) The results of method 1; (c) The results of method 2; (d) The results of method 3; (e) The results of method 4; (f) The results of method 5; (g) The results of the presented method. (Color figure online)

As shown in Fig. 8, aiming at insulator images with simple background, method 1 can’t remove wires and other false targets; method 2 can only remove the false targets with different color to insulator, and it is not applicable to other false targets; method 3 just removes the targets with different texture feature to insulator; method 4 can get good results, but it mostly relies on the amount of iteration and the manual selection of initial contour that will undoubtedly increase the user’s interaction; method 5 gets better results than other 4 methods, it uses undecimated wavelet feature and watershed algorithm, the method’ runtime is more long and can not remove false targets which adhere to insulator. The presented method uses the hybrid feature to get a better result.

As shown in Figs. 9 and 10, insulator’s texture feature is prominent for experimental images, method 3 is close to the presented method. But putting the information in Figs. 9 and 10 all together, the presented method is still better than the other 5 methods.

Fig. 9.

The comparison of the 6 segmentation methods’ IOU values.

Fig. 10.

The comparison of 6 segmentation methods’ Y values.

The presented method can get the best results. It generates hybrid feature by combining color feature with texture feature and uses relative modularity based detect communities to increase segmentation accuracy. Most importantly, it can remove false targets and background efficiently.

3.2 Segmentation Experiments About Aerial Insulator Images

We use the 6 methods to segment aerial insulator images with different background. The results of experiments about aerial insulator images are shown in Fig. 11.

Fig. 11.

The results of 6 segmentation methods. (a) Original image; (b) The results of method 1; (c) The results of method 2; (d) The results of method 3; (e) The results of method 4; (f) The results of method 5; (g) The results of the presented method. (Color figure online)

As shown in Fig. 11, the presented method can get the best segmentation results in these 6 methods, especially in aerial image (b).

In these methods, method 2 and method 3 only use single feature (color feature in method 2 and texture feature in method 3) to implement segmentation. But the super-pixels contain many characteristics, we can’t get ideal results when we just use single feature. Because method 2 and method 3 all use super-pixel method, their results are better than method 1, method 4 and method 5. Method 1 just removes small part of background, because the method can only get better segmentation results for the images with single peak between-class variance. For the images, method 4 and method 5 can remove most false targets. Method 4 can’t remove these wires due to the manual selection of initial contour, and it is a difficult job to select a contour that doesn’t contain wires. For method 5, the undecimated wavelet feature and watershed method lead to failure about removing false targets adhering to insulator. Because of the disadvantages of method 4 and method 5, the two methods almost can’t remove pole tower when we use them to segment aerial image (b) of Fig. 11(a).

Even if the processed images have complex background, the presented method can get the best results. It uses hybrid feature to calculate super-pixels’ similarity, which can find a little different between super-pixels. Then it uses community detection method to cluster super-pixels. Finally, super-pixels of insulators can be contained in a separate community. Most importantly, it can remove pole tower and wires adhering to insulators.

As shown in Figs. 12 and 13, method 2 has a little different with the presented method for segmenting aerial image (a) of Fig. 11(a). However, when we segment aerial image (b) of Fig. 11(a), advantages of the presented method become more obvious. The values of IOU and Y are all better than other 5 methods. Most importantly, the presented method can remove pole tower entirely and reserve all insulators.

Fig. 12.

The comparison of 6 segmentation methods IOU values.

Fig. 13.

The comparison of 6 segmentation methods Y values.

As shown in Figs. 9 and 10, the average IOU of the proposed method is 0.89 and the maximum of the other methods is 0.87. Moreover, the average Y of the proposed method is 0.56 and the maximum of the other methods is 0.54. Identically, in Figs. 11 and 12, the average IOU and Y of the proposed method are 0.83 and 0.62, but the maximum IOU and Y of the other methods are 0.55 and 0.43. They are far less than the proposed method. Therefore, no matter in Fig. 8, or in Fig. 11, the presented method can get the best results, especially in Fig. 11. These results show that the presented method is suitable to most insulator images, especially aerial images with pole tower.

3.3 Segmentation Experiments About Aerial Insulator Images with Noise

We add different noises into aerial insulator images. The noises we added are salt & pepper noise, Gaussian noise and speckle noise. Due to the low resolution of aerial images, we set the noise density of salt & pepper noise is 0.1, the variance of Gaussian noise is 0.01 and the variance of speckle noise is 0.1 (Fig. 14).

Fig. 14.

The segmentation results of different noise images processed by the proposed method. (a) Image with salt & pepper noise and Segmentation result; (b) Image with Gaussian noise and Segmentation result; (c) Image with speckle noise and Segmentation result.

The segmentation results of different noise images processed by the proposed method. (a) Image with salt & pepper noise and Segmentation result; (b) Image with Gaussian noise and Segmentation result; (c) Image with speckle noise and Segmentation result.

4 Conclusion

In this paper, we introduce the complex network community detection method into the insulator image segmentation. Based on complex network community and hybrid feature, we propose a segmentation method for aerial insulator image. It allows us to extract insulator as an entirety community from images with complex background. On the one hand, the TP super-pixel reduces the amount of the calculation. On the other hand, the complex network community method reduces the interference between different targets. And the insulator image is set up an entire community by using hybrid feature combined color feature and texture feature of each pixel. As long as the community is found, we can get all insulator information. Equally, other targets can be got, and we only need to find the corresponding communities.

Experimental results show that the presented method can efficiently segment insulator from images with many interference targets and different noises. Compared with the other 5 methods, the accuracy of the proposed method is more than 0.02 in IOU value and 0.02 in Y value for images with simple background and more than 0.28 in IOU value and 0.19 in Y value for images with complex background. The presented method is robust and has considerable accuracy and efficiency advantages. We achieve the automatic aerial insulator segmentation by the proposed segmentation method.