1 Introduction

Ensure the reliability of the transmission line is an important part of the smart grid construction. Insulators are indispensable element on transmission lines with the dual function of electrical insulation and wire support. Besides, frequent faults of insulators may lead to large-scale blackout and huge losses [1, 2]. One of the methods to improve the efficiency of insulator detection greatly is to process the transmission line inspection images by means of computer vision, and it can realize the request of intelligence and automation. Among them, the key and foundation of automatic detection is the localization of insulator in inspection images automatically [3].

Current trendy and top performing object detectors mostly employ region proposals to guide the detection and localization for objects [4, 5]. In image challenges based on world famous data set such as PASCAL VOC [6] and ImageNet [7], the object detectors which achieve outstanding performance and excellent effect all use the method of region proposals. For instance, the framework of R-CNN (Regions with CNN features) [8], which combined Selective Search [9] and CNN (Convolutional Neural Network) [10], has raised the detection rate from 35.1% to 53.7% in Pascal VOC Challenge. For an image, it first generates multiple region proposals and then, sets these proposals to fixed size to send to CNN for feature extraction and classification. Much follow-up work such as SPP-Net [11], Fast R-CNN [12] and Faster R-CNN [13] also involves the generation of region proposals, no matter in original images or feature maps obtained through deep neural network. Therefore, we settle down to finding a better generation method for region proposals, and it indeed has great significance in feature extraction and object localization.

In order to locate the insulators in transmission line inspection images automatically, and realize real-time detection and fault diagnosis on this basis, we choose the method of Edge Boxes [14], which has better performance in public data sets to be applied to transmission line inspection images that we obtained through professional equipment. However, the original intention of Edge Boxes is to detect the general objects in images, not specifically for insulators. If we expect to generate region proposals outstanding insulators only using Edge Boxes, the results are not satisfied. Therefore, in our method, we make full use of the prior characteristics of insulators and combine these characteristics with Edge Boxes. By processing the images with a series of operation, we finally generate the region proposals which contain more insulator parts. The experimental results show that our method can reduce the interference area effectively, and can ensure the follow-up phase of feature extraction to extract more pure insulator characteristics.

2 Related Work

2.1 Multiple-Scale Sliding Window

Inspired by the implementation of BING [15, 16], multiple-scale Sliding Window is the first one widely used for generating region proposals. Many fixed size windows slide in equidistant step on the images which are transformed into different scales. Due to the search space is the whole image, the greatest advantage is that its miss rate is extremely low and it will not leave any proposals out. However, Multiple-Scale Sliding Window classifiers increase linearly with the number of windows tested, and while single-scale detection requires classifying around 10\(^{4}\)–10\(^{5}\) windows per image, the number of windows grows by an order of magnitude for multi-scale detection. The huge search space and consumption of time further influenced the detection efficiency.

2.2 Selective Search

Instead of proposal generation method without any strategies like Multiple-Scale Sliding Window, Selective Search combines the strength of both exhaustive search [17] and segmentation. It uses the image structure to guide the sampling process like segmentation, meanwhile, it aims to capture all region proposals like exhaustive search. In Selective Search, a number of original areas are obtained by image segmentation, and then they are merged by strategy which based on color, texture and size. All object scales have to be taken into account. Compared with the traditional method with single strategy, Selective Search offered a variety of strategies, reduced the search space greatly and finally obtained more excellent results in object recognition. So far from Selective Search has been proposed, it has been widely used in many advanced object detection methods including R-CNN and Fast R-CNN. However, for the purpose of speeding up feature extraction, about 2000 region proposals generated from per image are still a stumbling block.

2.3 Edge Boxes

Edges provide a sparse but informative representation of an image. On the study of region proposals, Zitnick and Dollár [14] found a new way to generate object bounding box proposals which only use edges in an image. Edge Boxes took full advantage of the rich edge information in images, proposed a simple box objectness scoring method based on the number of edges that exist in the box and the number of contours that overlap the box’s boundary. This novel method can reduce the number of generated region proposals effectively, meanwhile, the calculating speed and precision has improved greatly compared with Selective Search.

However, the assumption of object parameters such as shape and size are not suitable for insulators in Edge Boxes. There might be some omissions of insulators, or too much interference of other components. Therefore, improving the method of Edge Boxes to generate region proposals, which are more suitable for insulators, is an important content of this paper. We considering the characteristics of insulators’ edge images, took a series of operations such as K-means clustering on CSS (Curvature Scale Space) points and circle on insulator subclass, combined insulators’ characteristics with Edge Boxes to get better performance. We will describe our method in detail in Sect. 3.

3 Our Method

3.1 Framework

For the input inspection image, we first do some preprocessing including graying, threshold segmentation and remove of the redundancy small area. We extract the edge of images and the CSS points [18, 19] in edge images. These CSS points are clustered into two subclasses by K-means. Then, we find CSS points which lie on suspected insulator subclass (the subclass that might be insulators) according to some certain rules, and use these points as the centers to form a set number of circles. This step can increase the number of edges that exist in the box completely which locates in the insulator. We put the images back to Edge Boxes scoring system, and now, the score of proposal box which contains insulator will increase, so as to make the output of the proposals contain more insulator subclasses. The framework of our method is shown in Fig. 1.

Fig. 1.
figure 1

Framework of our insulator region proposal generation method based on Edge Boxes (Color figure online)

Full size image

3.2 Image Preprocessing

The process of graying and threshold segmentation towards insulator inspection images, transforming the original images into binary images, can realize the separation of foreground and background. Containing varieties of objects such as insulator strings, towers, wire and inspectors, it is also difficult to determine the position of insulators in foreground. Choosing the method of morphological filtering, we first operate the binary images with morphological erosion in order to separate objects at slender points and remove the noise of tiny areas. Then the operation of morphological dilation fills the internal holes and smoothes the larger objects’ boundaries. These two morphological operations can remove most noise points, making the object edges smoother. As for the surviving small areas after filtering, we set a threshold to remove them. This step can eliminate the interference of impurity and improve the localization accuracy.

3.3 CSS Corners and K-Means Clustering

Contour curves [20] are extracted from the edge images. CSS corners are obtained as follows. Firstly, we calculate the curvature of each pixel point in contour curves under the high scale and choose the maximum curvature points as candidates for CSS corners. Secondly, if the curvature of one candidate point is greater than the preset threshold then mark this candidate point as the correct CSS corner. Finally, pinpoint all the correct CSS corners under the low scale.

Insulator string contains numbers of umbrella plates which have similarity in shape, meanwhile the curvature of each umbrella plate’s edge is almost the same. This character makes the distribution of CSS corners extracted from insulators very uniform yet no evident regularity is found in other component such as tower, wire and inspectors. Figure 2 shows the distribution of CSS corners in insulator strings.

Fig. 2.
figure 2

CSS corners in insulator strings. We can clearly see that these points distribute neatly and evenly on the edge and the joint of umbrella plates

Full size image
Fig. 3.
figure 3

Find suspected insulation subclasses by judging d (Color figure online)

Full size image

We cluster all CSS corners into two subclasses through K-means. Between each subclass, we find point A to represent the point which has the smallest abscissa and point B to represent the point which has the biggest abscissa. O is the center of clustering and if A and B has similar distance d towards O, we deem it be the suspected insulation subclass. To make the distinction results more accurate, we further considered the ordinate. The blue points in Fig. 3 are the CSS corners we found in suspected insulation subclass.

3.4 Circle

In Edge Boxes, one observation is that the more contours wholly contained in a bounding box, the more likely it contains an object. For this grading rule, we believe that if we can increase the number of contours around suspected insulation subclasses, so that we can improve box score which contains insulators. To make things easier, we adopt the method of CIRCLE. To be specific, centered on the blue CSS corners we found in Sect. 3.3, we circle numbers of circles with minor radius and these circles can increase closed contours effectively. Schematic diagram can be seen in the last two steps in Fig. 1.

4 Experimental Result

Based on several experiments, we have verified that our method has better performance compared with Sliding Window, Selective Search and pure Edge Boxes. The comparison unfolds from three aspects: effectiveness, precision and speed.

4.1 Effectiveness and Precision

In this section, we put the ratio of region proposals which contain insulator as an evaluation criterion of our method, and it can reflect the effectiveness when generating insulator region proposals:

$$\begin{aligned} \text {effectiveness} = \frac{ {{proposals\, contain\, insulators}}}{{{all\, proposals}}} \times 100 \% \end{aligned}$$
(1)

In addition to insulators, the proposals we generated usually also contain some background such as sky. Generally speaking, we hope that the area of insulator in the whole region proposal is as far as possible big and the area of background is as far as possible small. We define the index of precision to reflect the area ratio of groundtruth and proposals contain insulators:

$$\begin{aligned} \text {precision} = \frac{{{S_{groundtruth}}}}{{{S_{proposals\, contain \, insulators}}}} \times 100 \% \end{aligned}$$
(2)
Table 1. Effectiveness of different methods
Full size table
Table 2. Precision of different methods
Full size table
Table 3. Comparison of proposals generation speed
Full size table

Tables 1 and 2 give the effectiveness and precision of Sliding Window, Selective Search, Edge Boxes and our method (due to limited space, we present the experimental results on three images under each method). Table 1 shows the number of region proposals which contain insulators. Table 2 shows the ratio of all proposals when the precision reached 50%, 75% and 90% respectively. As we can see from the data in Tables 1 and 2, the number of proposals generated through pure Edge Boxes and our method has dropped a lot compared with Sliding Window and Selective Search. Specially, when employing the pure Edge Boxes, few box contains insulators was detected, sometimes none. However, through employing the method we proposed in this paper, all indexes have greatly increased, no matter the quantity or the quality of region proposals. A more intuitive comparison can be seen in Fig. 6.

Fig. 4.
figure 4

Parts of the region proposals generated through our method

Full size image
Fig. 5.
figure 5

Region proposals generated through different methods. (a) (d) Selective Search. (b) (e) pure Edge Boxes. (c) (f) Our method

Full size image

Figure 4 shows parts of the generated region proposals through our method. Figure 5 shows the proposals generated through three different methods: (a) (d) Selective Search, (b) (e) pure Edge Boxes and (c) (f) the method proposed in this paper. Hundreds proposals are generated through Selective Search and these waste too much time. The number of proposals generated through pure Edge Boxes and our method has dropped a lot, meanwhile, proposals generated through our method are more concentrated around the insulators.

4.2 Speed

In the task of object detection, we hope it consumes less time in the period of generating region proposals. In order to accelerate the whole process of object detection, the number of proposals needs to be cut down and the generation speed needs to be expedited at the same time. Table 3 shows the time that four methods need (identical to Sect. 4.1, we show results on three images and the present time is the average of ten experiments).

Fig. 6.
figure 6

Effectiveness and precision of different methods

Full size image

Among them, Sliding Window took far more time than three others because of the search space lies on the whole image. Compared with Selective Search, pure Edge Boxes increased substantially in generating speed and probably only about 1.69% to 4.15% of it. In this paper, our method achieved a slight acceleration or flat, for instance, the speed increased by 4.95% in test image 1 and 2.04% in test image No. 3.

5 Conclusion

In this paper, in the process of locating insulators in transmission line inspection images, we overcome the shortcoming that the proposal areas failed to highlight the insulators, and propose a generation method of insulator region proposals based on Edge Boxes. We considering the characteristics of insulators’ edge images, took a series of operations such as K-means clustering on CSS points and circle on insulator subclass, combined insulators’ characteristics with Edge Boxes to get better performance. With this method we proposed, more insulator region proposals are displayed and less interference regions are presented. Furthermore, the experiment results showed that our method did well both in effectiveness and precision, and achieved fast computation speed at the same time.