Skip to main content

Advertisement

Log in

Shuff-BiseNet: a dual-branch segmentation network for pavement cracks

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In order to accurately obtain the shape and size of pavement cracks, analyze the severity of pavement cracks, avoid deterioration of the situation, and take timely measures, we proposed a dual-branch structure Shuff-BiseNet road crack segmentation algorithm. Considering the slow speed of detail branch feature extraction, we introduced the basic unit of ShuffleNet to improve the speed of feature extraction and reduce the amount of parameters. Considering the problem of similar pavement cracks and pavement background characteristics, we introduced a variety of attention mechanisms to improve the network’s attention to the pavement crack area and effectively alleviate the impact of the background environment on the segmentation results. Taking into account the problem of pavement crack feature loss caused by using DWConv in the semantic branch, we introduced GSConv to solve the problem of feature loss and improve the receptive field through larger convolution kernel and residual structure. In order to avoid information loss caused by upsampling and downsampling processes and enhance feature expression capabilities, we replaced the pooling stage in the dual-branch feature fusion layer and introduced the DTSConve upsampling module and STDConv downsampling module. On the pavement crack segmentation data set, it achieved accuracy of 97.34%, mPA of 84.34%, IoU (crevice) of 54.81%, and mIoU of 76.03%. Compared with BiseNetv2, mPA, IoU (crevice), and mIoU increased by 4.22%, 4.22%, 2.81%, and 1.36%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

In this experiment, CrackForest was selected; the data set was enhanced and added. The datasets during the current study are not publicly available but are available from the corresponding author on reasonable request.

References

  1. Zou, C., Sun, Z., Cai, D., et al.: Crack detection using serrated columnar phased array transducers. Insight Non-Destruct. Test. Cond. Monit. 60(4), 212–219 (2018)

    Article  Google Scholar 

  2. Du, G., Huo, L., Kong, Q., et al.: Damage detection of pipeline multiple cracks using piezoceramic transducers. J. Vibroeng. 18(5), 2828–2838 (2016)

    Article  Google Scholar 

  3. Zima, B., Kdra, R.: Detection and size estimation of crack in plate based on guided wave propagation. Mech. Syst. Signal Process. 142, 106788 (2020)

    Article  Google Scholar 

  4. Chen, Y., Liang, J., Gu, X., et al.: An improved minimal path selection approach with new strategies for pavement crack segmentation. Measurement 184, 109877 (2021)

    Article  Google Scholar 

  5. Amhaz, R., Chambon, S., Idier, J., et al.: Automatic crack detection on two-dimensional pavement images: an algorithm based on minimal path selection. IEEE Trans. Intell. Transp. Syst. 17(10), 2718–2729 (2016)

    Article  Google Scholar 

  6. Kaddah, W., Elbouz, M., Ouerhani, Y., et al.: Optimized minimal path selection (OMPS) method for automatic and unsupervised crack segmentation within two-dimensional pavement images. Vis. Comput. 35(9), 1293–1309 (2019)

    Article  Google Scholar 

  7. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

  8. Wang, Z., Gao, X., Wu, R., et al.: Fully automatic image segmentation based on FCN and graph cuts. Multimedia Syst. 28(5), 1753–1765 (2022)

    Article  Google Scholar 

  9. Zhang, K., Shi, Y., Hu, C., et al.: Nucleus image segmentation method based on GAN and FCN model. Soft Comput. 26, 7449–7460 (2021)

    Article  Google Scholar 

  10. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18, Springer, pp. 234–241 (2015)

  11. Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., et al.: Unet++: a nested u-net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4, Springer, Berlin, pp. 3–11 (2018)

  12. Fu, L., Li, S.: A new semantic segmentation framework based on UNet. Sensors 23(19), 8123 (2023)

    Article  Google Scholar 

  13. Yang, Y., Dasmahapatra, S., Mahmoodi, S.: Ads_unet: a nested UNet for histopathology image segmentation. Expert Syst. Appl. 226, 120128 (2023)

    Article  Google Scholar 

  14. Al-Huda, Z., Peng, B., Algburi, R.N.A., et al.: A hybrid deep learning pavement crack semantic segmentation. Eng. Appl. Artif. Intell. 122, 106142 (2023)

    Article  Google Scholar 

  15. Chen, L.C., Papandreou, G., Kokkinos, I., et al.: Semantic image segmentation with deep convolutional nets and fully connected crfs. (2014) arXiv preprint arXiv:1412.7062

  16. Chen, L.C., Papandreou, G., Kokkinos, I., et al.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  17. Chen, L.C., Zhu, Y., Papandreou, G., et al.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 801–818 (2018)

  18. Fu, H., Meng, D., Li, W., et al.: Bridge crack semantic segmentation based on improved deeplabv3+. J. Mar. Sci. Eng. 9(6), 671 (2021)

    Article  Google Scholar 

  19. Yuan, H., Zhu, J., Wang, Q., et al.: An improved deeplab v3+ deep learning network applied to the segmentation of grape leaf black rot spots. Front. Plant Sci. 13, 795410 (2022)

    Article  Google Scholar 

  20. Xie, E., Wang, W., Yu, Z., et al.: Segformer: simple and efficient design for semantic segmentation with transformers. Adv. Neural. Inf. Process. Syst. 34, 12077–12090 (2021)

    Google Scholar 

  21. Cao, H., Wang, Y., Chen, J., et al.: Swin-Unet: Unet-like pure transformer for medical image segmentation. In: European Conference on Computer Vision, Springer, pp. 205–218 (2022)

  22. Liu, H., Yang, J., Miao, X., et al.: Crackformer network for pavement crack segmentation. IEEE Trans. Intell. Transp. Syst. (2023). https://doi.org/10.1109/TITS.2023.3266776

    Article  Google Scholar 

  23. Wang, W., Su, C.: Automatic concrete crack segmentation model based on transformer. Autom. Constr. 139, 104275 (2022)

    Article  Google Scholar 

  24. Zhang, T., Wang, D., Lu, Y.: Ecsnet: an accelerated real-time image segmentation CNN architecture for pavement crack detection. EEE Trans. Intell. Transp. Syst. (2023)

  25. Hou, Y., Liu, S., Cao, D., et al.: A deep learning method for pavement crack identification based on limited field images. IEEE Trans. Intell. Transp. Syst. 23(11), 22156–22165 (2022)

    Article  Google Scholar 

  26. Li, S., Gu, X., Xu, X., et al.: Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm. Constr. Build. Mater. 273, 121949 (2021)

    Article  Google Scholar 

  27. Zhang, T., Wang, D., Mullins, A., et al.: Integrated APC-GAN and AttuNet framework for automated pavement crack pixel-level segmentation: a new solution to small training datasets. IEEE Trans. Intell. Transp. Syst. 24(4), 4474–4481 (2023)

    Article  Google Scholar 

  28. Yu, C., Wang, J., Peng, C., et al.: Bisenet: bilateral segmentation network for real-time semantic segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 325–341 (2018)

  29. Yu, C., Gao, C., Wang, J., et al.: Bisenet v2: bilateral network with guided aggregation for real-time semantic segmentation. Int. J. Comput. Vis. 129, 3051–3068 (2021)

    Article  Google Scholar 

  30. Ma, N., Zhang, X., Zheng, H.T., et al.: Shufflenet v2: practical guidelines for efficient CNN architecture design. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 116–131 (2018)

  31. Li, H., Li, J., Wei, H., et al.: Slim-neck by GSConv: a better design paradigm of detector architectures for autonomous vehicles (2022) arXiv preprint arXiv:2206.02424

  32. Han, K., Wang, Y., Tian, Q., et al.: GhostNet: more features from cheap operations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1580–1589 (2020)

  33. Howard, A., Sandler, M., Chu, G., et al.: Searching for mobilenetv3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1314–1324 (2019)

  34. Ma, N., Zhang, X., Zheng, H.T., et al.: Shufflenet v2: practical guidelines for efficient CNN architecture design. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 116–131 (2018)

  35. Mehta, S., Rastegari, M., Shapiro, L., et al.: Espnetv2: a light-weight, power efficient, and general purpose convolutional neural network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9190–9200 (2019)

  36. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  37. Wang, Q., Wu, B., Zhu, P., et al.: Eca-net: efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11534–11542 (2020)

  38. Woo, S., Park, J., Lee, J.Y., et al.: Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  39. Liu, M., Zhang, F., Sun, X., et al.: Ca-net: leveraging contextual features for lung cancer prediction. In: Medical Image Computing and Computer Assisted Intervention—MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part V 24, Springer, pp. 23–32 (2021)

  40. Sun, K., Xiao, B., Liu, D., et al.: deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5693–5703 (2019)

  41. Zhao, H., Shi, J., Qi, X., et al.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

Haiqun mainly completed the writing of the thesis, Bingnan mainly completed the improvement in the model and the ablation experiment and comparison experiment, and Tao mainly completed the comparison of experimental data and the conclusion of the paper

Corresponding author

Correspondence to Haiqun Wang.

Ethics declarations

Ethical approval

Not applicable

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Wang, B. & Zhao, T. Shuff-BiseNet: a dual-branch segmentation network for pavement cracks. SIViP 18, 3309–3320 (2024). https://doi.org/10.1007/s11760-023-02993-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02993-2

Keywords

Navigation