Skip to main content
SpringerLink
Log in

A neural network ensemble method for effective crack segmentation using fully convolutional networks and multi-scale structured forests

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Crack image segmentation has recently become a major research topic in nondestructive inspection. However, the image segmentation methods are not robust to variations such as illumination, weather, noise and the segmentation accuracy which cannot meet the requirements of practical applications. Therefore, a neural network ensemble method is proposed for effective crack segmentation in this paper, which consists of fully convolution networks (FCN) and multi-scale structured forests for edge detection (SFD). In order to improve the accuracy of crack segmentation and reduce the error mark under complex background, a new network model based on FCN model is proposed to address the problems that lose local information and the capacity of partial refinement, which are frequently encountered in FCN model in the crack segmentation. In addition, SFD is combined with the half-reconstruction method of anti-symmetrical bi-orthogonal wavelet to overcome the limitation of crack edge detection. Finally, the result of the two maps is merged after resizing to the original image dimensions. Qualitative and quantitative evaluations of the proposed methods are performed, showing that they can obtain better results than certain existing methods for crack segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Luo, Q.W., He, Y.G.: A cost-effective and automatic surface defect inspection system for hot-rolled flat steel. Robot. Comput.-Integr. Manuf. 38(3), 16–30 (2016)

    Google Scholar 

  2. Kehoe, A., Parker, G.A.: Image processing for industrial radiographic inspection: image enhancement. Br. J. NDT 32(4), 183–190 (1990)

    Google Scholar 

  3. Amet, L.L., Ertuzun, A., Ercil, A.: An efficient method for texture defect detection: subband domain co-occurrence matrices. Image Vis. Comput. 18(6–7), 543–553 (2000)

    Google Scholar 

  4. Bodnarova, A., Bennamoun, M., Latham, S.: Optimal gabor filters for textile flaw detection. Pattern Recognit. 35(12), 2973–2991 (2002)

    MATH  Google Scholar 

  5. Wiltschi, K., Pinz, A., Lindeberg, T.: An automatic assessment scheme for steel quality inspection. Mach. Vis. Appl. 12(3), 113–128 (2000)

    Google Scholar 

  6. Clelia, M., Ettore, S., Massimiliano, N., Nicola, A., Arcangelo, D.: Rail corrugation detection by Gabor filtering. In: IEEE International Conference on Image Processing, pp. 626–628 (2001)

  7. Kasban, H., Zahran, O., Arafa, H., El-Kordy, M., ELARABY, S.M.S., Abd EI-Samie, F.E.: Welding defect detection from radiography images with a cepstral approach. NDT & E Int. 44, 226–231 (2011)

    Google Scholar 

  8. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017)

    Google Scholar 

  9. Piotr, D., Lawrence, Z.C.: Fast edge detection using structured forests. IEEE Trans. Pattern Anal. Mach. Intell. 37(8), 1558–1570 (2015)

    Google Scholar 

  10. Chi, J.N., Zhang, C., Zhang, Z.H.: Image enhancement based on antisymmetrical bioryhogonal wavelet recontruction. Acta Autom. Sin. 36(4), 475–487 (2010)

    Google Scholar 

  11. Martnez, S.S., Ortega, J.G., Garcia, J.G., Garcia, A.S.: A machine vision system for defect characterization on transparent parts with non-plane surfaces. Mach. Vis. Appl. 23(1), 1–13 (2012)

    Google Scholar 

  12. He, Y.Q., Qiu, H.X.: A method of cracks image segmentation based on the means of multiple thresholds. J. Commun. Comput. 9, 1147–1151 (2012)

    Google Scholar 

  13. Kima, H., Ahna, E., Chob, S.: Comparative analysis of image binarization methods for crack identification in concrete structures. Cem. Concr. Res. 99, 53–61 (2017)

    Google Scholar 

  14. Li, Y.Y., Sagar, D., Peng, Y.K.: A machine vision system for identification of micro-crack in egg shell. J. Food Eng. 109, 127–134 (2012)

    Google Scholar 

  15. Aswini, E., Divya, S., Kardheepan, S., Manikandan, T.: Mathematical morphology and bottom-hat filtering approach for crack detection on relay surfaces. In: IEEE International Conference on Smart Structures & Systems, pp. 108–113 (2013)

  16. Jetsadaporn, P., Chawalit, K., Somying, T.: Crack detection on unwashed eggs using image processing. J. Food Eng. 209, 76–82 (2017)

    Google Scholar 

  17. Bharathi, S.S., Radhakrishnan, N., Priya, L.: Surface defect detection of rubber oil seals based on texture analysis. In: IEEE 4th International Conference on Signal and Image Processing, pp. 207–216 (2012)

  18. Withawat, W., Pichate, K., Channarong, A., Pitikhate, S.: Rapid detection of hairline cracks on the surface of piezoelectric ceramics. Int. J. Adv. Manuf. Technol. 64, 1275–1283 (2013)

    Google Scholar 

  19. Yusuke, F., Yoshihiko, H.: A robust automatic crack detection method from noisy concrete surfaces. Mach. Vis. Appl. 22, 245–254 (2011)

    Google Scholar 

  20. Yang, C.L., Liu, P.Y., Yin, G.F., Wang, L.: Crack detection in magnetic tile images using nonsubsampled shearlet transform and envelope gray level gradient. Opt. Laser Technol. 90, 7–17 (2017)

    Google Scholar 

  21. Yamaguchi, T., Hashimoto, S.: Fast crack detection method for large-size concrete surface images using percolation-based image processing. Mach. Vis. Appl. 21, 797–809 (2010)

    Google Scholar 

  22. Fadel, M.M., Jaime, A.C.: Real-time fault detection in manufacturing environments using face recognition techniques. J. Intell. Manuf. 23, 393–408 (2012)

    Google Scholar 

  23. Cinara, A.F., Barhlib, S.M., Hollis, D., Flansbjer, M., Tomlinson, R.A., Marrow, T.J., Mostafavi, M.: An autonomous surface discontinuity detection and quantification method by digital image correlation and phase congruency. Opt. Lasers Eng. 96, 94–106 (2017)

    Google Scholar 

  24. Zou, Q., Cao, Y., Li, Q.Q., Mao, Q.Z., Wang, S.: CrackTree: automatic crack detection from pavement images. Pattern Recognit. Lett. 3, 227–238 (2012)

    Google Scholar 

  25. Mohamad, M.H., Satya, M.M., Desire, S., Fabrice, M.: Multiple features extraction for timber defects detection and classification using SVM. In: IEEE International Conference on Image Processing, pp. 427–431 (2015)

  26. Suvdaa, B., Ahn, J., Ko, J.: Steel Surface defects detection and classification using SIFT and voting strategy. Int. J. Softw. Eng. Appl. 6(2), 161–165 (2012)

    Google Scholar 

  27. Hu, H.J., Liu, Y., Liu, M.F., Nie, L.Q.: Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm. Neurocomputing 181, 86–95 (2016)

    Google Scholar 

  28. Sukalpa, C., Bu, G.P., Guan, H., Jun, J., Umapada, P., Yew-Chaye, L., Michael, B.: Automatic bridge crack detection—a texture analysis-based approach. In: IAPR Workshop on Artificial Neural Networks in Pattern Recognition, pp. 193–203 (2014)

  29. Li, G., Zhao, X.X., Du, K., Ru, F., Zhang, Y.B.: Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine. Autom. Constr. 78, 51–61 (2017)

    Google Scholar 

  30. Hoang, N., Nguyen, Q., Bui, D.T.: Image processing-based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony. J. Comput. Civ. Eng. 32(5), 04018037 (2018)

    Google Scholar 

  31. Banharnsakun, A.: Hybrid ABC-ANN for pavement surface distress detection and classification. Int. J. Mach. Learn. Cybern. 8, 699–710 (2017)

    Google Scholar 

  32. Cubero-Fernandez, A., Rodriguez-Lozano, F.J., Rafael, V., Joaquin, O., Jose, M.P.: Efficient pavement crack detection and classification. EURASIP J. Image Video Process. 2017, 39–49 (2017)

    Google Scholar 

  33. Chen, S.G., Lin, B., Han, X.S., Liang, X.H.: Automated inspection of engineering ceramic grinding surface damage based on image recognition. Int. J. Adv. Manuf. Technol. 66, 431–443 (2013)

    Google Scholar 

  34. Shi, P.F., Fan, X.N., Wang, G.R.: A novel underwater dam crack detection algorithm based on sonar images. In: Proceedings of the 5th International Conference on Computer Sciences and Automation Engineering, pp. 452–456 (2015)

  35. Meng, F., Li, A.: Pavement crack detection using sketch token. Procedia Comput. Sci. 139, 151–157 (2018)

    Google Scholar 

  36. Luo, Q.J., Ge, B.Z., Tian, Q.G.: A fast adaptive crack detection algorithm based on a double-edge extraction operator of FSM. Constr. Build. Mater. 24, 244–254 (2019)

    Google Scholar 

  37. Cha, Y.J., Choi, W.: Deep learning-based crack damage detection using convolutional neural networks. Comput.-Aided Civ. Infrastruct. Eng. 32, 361–378 (2017)

    Google Scholar 

  38. Zhang, L., Yang, F., Zhang, Y.D., Zhu, Y.J.: Road crack detection using deep convolutional neural network. In: IEEE International Conference on Image Processing, pp. 3708–3712 (2016)

  39. Gopalakrishnan, K., Khaitan, S.K., Choudhary, A., Agrawal, A.: Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection. Constr. Build. Mater. 157, 322–330 (2017)

    Google Scholar 

  40. Zhang, A., Wang, K.C.P.: Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Comput.-Aided Civ. Infrastruct. Eng. 32, 805–819 (2017)

    Google Scholar 

  41. Zhang, A., Wang, K.C.P., Fei, Y., Liu, Y., Tao, S.Y., Chen, C., Li, J.Q., Li, B.X.: Deep learning-based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet. J. Comput. Civ. Eng. 32(5), 04018041 (2018)

    Google Scholar 

  42. Chen, F.C., Jahanshahi, M.R.: NB-CNN: deep learning-based crack detection using convolutional neural network and Naive Bayes data fusion. IEEE Trans. Ind. Electron. 65(5), 4392–4400 (2018)

    Google Scholar 

  43. Liu, H., Guo, R.Y.: Detection and identification of SAWH pipe weld defects based on X-ray image and CNN. Chin. J. Sci. Instrum. 39(4), 247–256 (2014)

    Google Scholar 

  44. Chang, H.T., Gou, J.N., Li, X.M.: Application of faster R-CNN in image defect detection of industrial CT. J. Image Graph. 23(7), 1061–1071 (2018)

    Google Scholar 

  45. Huang, H.W., Li, Q.T., Zhang, D.M.: Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunn. Undergr. Space Technol. 77, 166–176 (2018)

    Google Scholar 

  46. Dung, C.V., Anh, L.D.: Autonomous concrete crack detection using deep fully convolutional neural network. Autom. Construct. 99, 52–58 (2019)

    Google Scholar 

  47. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)

    Google Scholar 

  48. He, K., Zhang, X.Y., Ren, S.Q., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  49. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder–decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Google Scholar 

  50. Wang, S., Wu, X., Zhang, Y.H., Chen, Q.: Image crack detection with multi-scale down-sample normalized cut. Chin. J. Sci. Instrum. 38(11), 2788–2796 (2017)

    Google Scholar 

  51. Boykov, Y., Kolmogorov, V., Zhang, D.M.: A experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)

    Google Scholar 

  52. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Publishing House of Electronics Industry, Beijing (2011)

    Google Scholar 

  53. Perazzi, F., Krahenbuhl, P., Pritch, Y., Hornung, A.: Saliency filters: contrast based filtering for salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 733–740 (2012)

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under (51875272, 61761024), the Talent Development Project of Yunnan Province under (KKSY201801018), Yunnan Provincial Department of Education Science Research Fund Project under (2019J0045) and the Introduction Talents Project of Guizhou University under (2018(18)).

Author information

Authors and Affiliations

  1. Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Yunnan, 650500, China

    Sen Wang, Xing Wu, Yinghui Zhang & Xiaoqin Liu

  2. Yunnan Vocational College of Mechanical and Electrical Technology, Yunnan, 650203, China

    Xing Wu

  3. School of Mechanical Engineering, Guizhou University, Guizhou, 550025, China

    Lun Zhao

Authors
  1. Sen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xing Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yinghui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoqin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xing Wu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Wu, X., Zhang, Y. et al. A neural network ensemble method for effective crack segmentation using fully convolutional networks and multi-scale structured forests. Machine Vision and Applications 31, 60 (2020). https://doi.org/10.1007/s00138-020-01114-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01114-0

Keywords

Search

Navigation