Keywords

1 Introduction

Concrete architectures are the most common infrastructure in the city. After years of service, the occurrence of cracks in different degrees will harm the safety of concrete buildings. Therefore, the accurate detection of cracks is of great significance to the evaluation and restoration of structures. So far, there have been a variety of crack detection methods, such as the artificial detection, the ultrasonic detection and the machine vision detection [1, 2]. The artificial detection is dependent on the experienced workers, which is time consuming and gradually replaced by the visual based methods. The visual based detection methods have aroused the interest of the researchers [1,2,3,4].

Over the past few decades, a large number of crack detection algorithms based on image processing have been proposed in the literature. According to the difference of the information used by the approaches, these models are roughly divided into the following categories: intensities based methods [5, 6], edge based methods [7, 8], shape based methods [9, 10] and the integrated methods [10, 11]. Normally, the cracks in the structure images are mainly characterized as follows: (1) The pixels intensities belonging to the crack are darker than the ones of the background; (2) The local crack patches are approximated as linear, which distribution contains longitudinal, transverse, diagonal and the like [9]; (3) The pixel number of the crack is much less than the one of the background.

Initially, the researchers take the gray level information into consideration to find the thresholding value between the crack target and the background. However, these methods are affected by the noise and lack the global constraints, which result in poor detection results. Since the differences in cracks and background grayscale are distinct, the gradient based edges are the important features for crack detection. Hence, the edge detection algorithms such as the Canny and Sobel are used to deal with different types of surface cracks [8, 12]. Similarly, the edge detection models are easily effected by the noise and the complex background. To address these problems, more feature information is utilized to guide the crack segmentation, such as the geometric features and the textures [13, 14]. Given the linear structures of the crack, the Hessian filter, which is developed for the vessel enhancement [16], is employed to emphasize the characteristics of the crack [15]. The drawback of the model is that serious inhomogeneity would easily lead to error detection since the step edges would be taken as crack.

Meanwhile, machine learning based methods have been another important approaches for crack detection. For this kind of methods, the 2-D geometric characteristics such as the longitudinal, transverse, diagonal, are as the input features of the classifiers [9, 17, 18]. The supervised classifiers, for example the SVM [17], the neural network [18] are used to classify the crack and non-crack. However, the techniques work on every pixel or local sub-image, and therefore lack global constraints. Further, some model-based crack segmentation methods are proposed [10, 19, 20] in the field. The active contour model (ACM) [11, 20], which objective function is constructed by the features of the crack (intensity, edges, shape, etc.), is considered as a general framework. In general, The existing problems of aforementioned models could be solved to certain extent by the ACM. For example, the Region-Scalable Fitting (RSF) model and the like, which are put forward in [21, 22], work in spatially varying local region, and hence segment images with intensity inhomogeneity effectively. The model is subjected to the global constraints of the length and area, so it is robust to noise.

In this paper, a joint crack detection model is proposed. Firstly, the nonlocal mean filtering is employed to reduce the noise which appears during the acquisition of the concrete architecture image, preserving the details of the edges information simultaneously. Secondly, an improved multi-scale linear feature enhancement filtering is used to strengthen the crack target; Then, a set of morphological operations and the thresholding model are employed to ameliorate the results and output a binary image which is used to initialize the level sets and guide the evolution of the active contours. Finally, the localized active contour model integrating the intensity and the shape information is utilized to refine the coarse results.

The rest of the paper is arranged as follows: Sect. 2 provides the detailed description of the whole algorithm; experiments and analysis are carried out in Sect. 3; finally, the conclusion is drawn in Sect. 4.

2 The Proposed Model

In the section, the proposed automatic crack detection is described in detail. The whole flowchart of the proposed model is shown in Fig. 1. The whole framework can be divided into four parts: In the first part, the nonlocal mean denoising filter is employed to smooth the crack image while preserve the object edge information. In the second part, the improved multi-scale Hessian-based filter is used to enhance the dark line-like object. Meanwhile, two morphology operators: the dilation reconstruction and the erosion reconstruction, are introduced to remove the noise and the pseudo small targets in the image, which are followed by the Otsu thresholding to output a binary image. The binary image has two major roles: on the one hand, it works as the initial active contour; on the other hand, it works as the shape constraint term of the energy functional. The results of each step are shown in Fig. 2.

Fig. 1.
figure 1

Flowchart of the proposed model

Full size image
Fig. 2.
figure 2

The processing results of every step of the proposed model. (a) Crack image. (b) NLM filtered result. (c) Hessian based filtered result. (d) The result of the application of morphology opening reconstruction and closing reconstruction. (e) Thresholding result. (d) Final contours of the ACM. (g) The corresponding binary result determined by the final contours. (h) Postprocessing results (if necessary).

Full size image

2.1 Nonlocal Mean Image Denoising

The nonlocal mean image filter (NLM) [23] makes full use of the redundant information in the image, and maintains the details of the image while de-noising. The basic idea of the NLM filter lies in: the pixels in the image are weighted averagely by the pixels with similar structures in the neighborhood of the image. However, the NLM filter takes a lot of time to calculate the weight value, which limits its application. Later, some fast NLM models are proposed to accelerate the denoising process [24]. In this paper, the fast NLM filter in [24] is introduced to smooth the images (refer to [24] for details). The filtering results could be observed in Fig. 2(b), which demonstrates that the noise is effectively suppressed and the crack edges are well maintained simultaneously.

2.2 Hessian Based Multi-scale Filtering

The Hessian matrix calculates the second order information of the pixels \( \left( {x,y} \right) \) in the image \( I{:}\;\Omega \to \Re \).

$$ {\text{H}}\left( {x,\;y} \right) = \left[ {\begin{array}{*{20}c} {\frac{{\partial^{2} L}}{{\partial^{2} x}}} & {\frac{{\partial^{2} L}}{\partial x\partial y}} \\ {\frac{{\partial^{2} L}}{\partial y\partial x}} & {\frac{{\partial^{2} L}}{{\partial^{2} y}}} \\ \end{array} } \right] $$
(1)

Where, \( L = G\left( {x,y,\,s} \right) * I\left( {x,y} \right) \) denotes the convolution of the original image and the Gaussian function, where, \( s \) is the scale parameter. The eigenvalues \( \left( {\lambda_{1} ,\lambda_{2} ,\quad \left| {\lambda_{1} } \right| \le \left| {\lambda_{2} } \right|} \right) \) and the corresponding eigenvectors are important shape indices, and the larger eigenvalues represent the direction of the larger curvature. With these eigenvalues, the likelihood function is developed in [16] according to the two measures: \( {\text{R}}_{a} = {{\left| {\lambda_{1} } \right|} \mathord{\left/ {\vphantom {{\left| {\lambda_{1} } \right|} {\left| {\lambda_{2} } \right|}}} \right. \kern-0pt} {\left| {\lambda_{2} } \right|}} \), \( {\text{R}}_{b} = \sqrt {\left| {\lambda_{1} } \right|^{2} + \left| {\left| {\lambda_{2} } \right|^{2} } \right|} \). However, the response in [16] between cracks of different transverse radii is usually inhomogeneous. Given that, the cracks have complex textures, the likelihood function is designed in this paper as:

$$ {\text{V}}\left( {x,y,s} \right) = \left\{ {\begin{array}{*{20}c} {0,} & {if\;\lambda_{2} \le 0} \\ {e^{{\left( { - {{\left( {\lambda_{1} + \lambda_{2} } \right)} \mathord{\left/ {\vphantom {{\left( {\lambda_{1} + \lambda_{2} } \right)} {R_{b}^{2} }}} \right. \kern-0pt} {R_{b}^{2} }}} \right)}} \left( {1 - e^{{\left( { - {{R_{b}^{2} } \mathord{\left/ {\vphantom {{R_{b}^{2} } {2c}}} \right. \kern-0pt} {2c}}} \right)}} } \right)} & {\text{otherwise}} \\ \end{array} } \right.\quad $$
(2)

Where, \( c \) is the constant parameter to control the sensitivity of measures \( {\text{R}}_{b} \). Note that, different scales parameters lead to different response results, and the final result is the maximum response of all the responses.

After the NLM denoising, the enhancement result of the multi-scale Hessian based filtering is displayed in Fig. 2(c), which demonstrates the ability of the approach to detect the linear targets.

2.3 Coarse Segmentation

Meanwhile, some noise and other small linear noncracks are detected as shown in Fig. 2(c). Thus some subsequent processing approaches are used to obtain the binary image.

Morphology Processing.

The morphology opening operators with linear structures element could detect the crack in a certain shape. Thus the structures element with different angular rotations and sizes could detect all the linear shape. To eliminate the effects of noise, and preserve the small and tortuous crack, the supremum of the openings and the reconstruction operator [25] are adopted to deal with the result of the Hessian based enhancement. Then, the dual filter, which replaces the opening operator by the closing one, is used to fill in the hole of the result as in [25]. The result of the morphology operators could be seen in Fig. 2(d).

Otsu Thresholding.

Further, the Otsu thresholding is introduced to generate the binary image. It is assumed that the image pixels can be divided into two parts: the background and the objects according to the threshold. And the threshold is determined based on the maximum variance of the inter classes. The binary output mask is shown in Fig. 2(e), which could be considered as a coarse segmentation of the crack.

2.4 Localized Region Based ACM

The proposed localized region based ACM integrates the advantages of the LCV [22] and the multi-scale Hessian based enhancement. The LCV model uses the local image information to segment the images with intensity inhomogeneity effectively. The energy function consists of six terms: the first two terms are the fitting energy terms which measure the similarity between the image pixel and the local intensity mean; The third and fourth terms are the length term and the area term which smooth the zero level contours. The fifth term is the penalty term which penalizes the deviation of the level set function from the signed distance function. The last one is the shape constraint term, where the \( \varphi \left( {\text{x}} \right) \) is the signed distance function obtained mainly by the Hessian based denoising. The energy function is expressed as:

$$ \begin{aligned} E\left( {\phi \left( {\mathbf{x}} \right),\;f_{1} ,\;f_{2} } \right) & = \int {\left( {I - f{}_{1}\left( {\mathbf{x}} \right)} \right)}^{2} H\left( {\phi \left( {\mathbf{x}} \right)} \right)d{\mathbf{x}} + \int {\left( {I - f{}_{2}\left( {\mathbf{x}} \right)} \right)}^{2} \left( {1 - H\left( {\phi \left( {\mathbf{x}} \right)} \right)} \right)d{\mathbf{x}} \\ & + \,\alpha \int {\left| {\nabla H\left( {\phi \left( {\mathbf{x}} \right)} \right)} \right|} d{\mathbf{x}} + \beta \int {H\left( {\phi \left( {\mathbf{x}} \right)} \right)} d{\mathbf{x}} \\ & + \,\mu \int {\frac{1}{2}} \left| {\nabla \phi \left( H \right){ - }1} \right|^{2} d{\mathbf{x}} + \nu \int {\left( {H\left( {\phi \left( {\mathbf{x}} \right)} \right){ - }H\left( {\varphi \left( {\mathbf{x}} \right)} \right)} \right)}^{2} d{\mathbf{x}} \\ \end{aligned} $$
(3)

Where, \( \alpha ,\;\beta ,\,\mu ,\,\nu \) are the weight parameters of the energy terms. \( f_{1} \left( {\mathbf{x}} \right),\;\;f_{2} \left( {\mathbf{x}} \right) \) locally approximate the intensities on both sides of the zero level set curve, which are calculated as:

$$ f_{1} \left( {\mathbf{x}} \right) = \frac{{\int {K\left( {\mathbf{x}} \right)I\left( {\mathbf{x}} \right)H\left( {\phi \left( {\mathbf{x}} \right)} \right)d{\mathbf{x}}} }}{{\int {H\left( {\phi \left( {\mathbf{x}} \right)} \right)d{\mathbf{x}}} }},\;\;f_{2} \left( {\mathbf{x}} \right) = \frac{{\int {K\left( {\mathbf{x}} \right)I\left( {\mathbf{x}} \right)\left( {1{ - }H\left( {\phi \left( {\mathbf{x}} \right)} \right)} \right)d{\mathbf{x}}} }}{{\int {K\left( {\mathbf{x}} \right)\left( {1{ - }H\left( {\phi \left( {\mathbf{x}} \right)} \right)} \right)d{\mathbf{x}}} }} $$
(4)

Where, \( H\left( {\mathbf{x}} \right) \) is the Heaviside function which denotes the internal region of the active curve while \( \left( {1{ - }H\left( {\mathbf{x}} \right)} \right) \) represents the external region. \( K \) is the Gaussian kernel function. Minimize the functional of (3), we obtain the corresponding Euler-Lagrange equation:

$$ \begin{aligned} \frac{\partial \phi }{\partial t} & = - \delta \left( \phi \right)\left[ {\left( {I - f{}_{2}\left( {\mathbf{x}} \right)} \right)^{2} - \left( {I - f{}_{1}\left( {\mathbf{x}} \right)} \right)^{2} } \right] + \alpha \delta \left( \phi \right)div\left( {\frac{\nabla \phi }{{\left| {\nabla \phi } \right|}}} \right) + \beta \delta \left( \phi \right) \\ & + \,\mu \left( {\Delta \phi - div\left( {\frac{\nabla \phi }{{\left| {\nabla \phi } \right|}}} \right)} \right) + \nu \delta \left( \phi \right)\left( {H\left( {\phi \left( {\mathbf{x}} \right)} \right){ - }H\left( {\varphi \left( {\mathbf{x}} \right)} \right)} \right) \\ \end{aligned} $$
(5)

Where, \( \delta \left( \phi \right) \) is the Dirac function. In this paper, the initialized level set function is obtained by:

$$ \phi_{0} \left( {\mathbf{x}} \right) = \varphi \left( {\mathbf{x}} \right) = 4 \times \left( {0.5 - {\text{B}}\left( {\mathbf{x}} \right)} \right) $$
(6)

Where, the mask \( {\text{B}}\left( {\mathbf{x}} \right) \) is the binary result of the thresholding segmentation. Thus the difference between the constant 0.5 and the mask could obtain the different signs on both sides of the contour. Thus the initialization is performed automatically.

3 Experiments

In this section, several experiments are carried out to show the effectiveness of the proposed approach. All the experiments are conducted in MATLAB R2015a, on a personal computer.

First, we compare several filters that can protect the edges while denoising. A lot of fine particles in the concrete building images interfere with the cracks, which would further have a negative impact on crack detection. The bilateral filter, the guided filter and the NLM filter used in this paper are carried out on the original crack images, which can be seen in Fig. 3(b)–(d).

Fig. 3.
figure 3

The denoising results using three filters

Full size image

Using the guided filter, the noise is suppressed and hence it provides the most smoothest background. Meanwhile, the cracks are largely blurred as shown in Fig. 3(c). On the contrary, the bilateral filter shows better performance than the guide filter, which reduces some noise in the background and keeps the target edge to some extent, as shown in Fig. 3(b). Among all the evaluated models, the NLM filter obtain the best performance, providing the clear edges and the similar background to the guided filter as shown in Fig. 3(d).

Then, we demonstrate the ability of the improved Hessian based enhancement model in this paper. The classical Frangi’s model [16] is proved to be an effective method of linear object enhancement, which is shown in Fig. 4(b). However, the response of the Frangi’s model is lower at the transition and bifurcations, which are marked with yellow circles in Fig. 4(b). Consequently, small cracks could not be extracted by the detection. On the contrary, the improved model is able to enhance all the cracks, which is clearly seen in Fig. 4(c).

Fig. 4.
figure 4

The enhancement results using two kinds of Hessian based models

Full size image

The experiments on several crack images in Fig. 5 are conducted to compare the proposed model to the state-of-art ACM models, i.e. the GC model [20], and the LCV model [22]. The GC model (Geodesic active contours model) is based on the gradient and curvature to detect the crack boundary. The precision and speed of the ACM models depend greatly on the initial contour curve. And one need to set the initial contours carefully to get better performance. Thus, we set the same initialization obtained by the Hessian based enhancement and the thresholding for the GC model and the LCV model, which could be seen in Fig. 5(a).

Fig. 5.
figure 5

Results of different ACMs on crack images.

Full size image

Figure 5(a) reveals that, when the crack is simple, all the methods can detect the crack successfully, benefiting from the initialization technique. As the crack becomes complex, the segmentation results of the GC model get worse. However, the LCV model provides satisfactory results, despite some unsmooth and abrupt edges. The proposed method achieves the best performance for all images, confirming that it is an effective approach for crack detection.

4 Conclusion

Our main contribution is to present a hybrid framework for crack detection, which integrates the advantages of the NLM filter, the Hessian based enhancement and the localized ACM. Both the intensity and the shape information of the crack is utilized to construct the level set functional. Also, the automatic initialization of the active contour is realized through the results based on the Hessian-based enhancement. Some experiments have demonstrated the functions of each part of the hybrid framework. The proposed approach has the potential to be an effective way for crack detection.