Abstract
In order to accurately obtain the shape and size of pavement cracks, analyze the severity of pavement cracks, avoid deterioration of the situation, and take timely measures, we proposed a dual-branch structure Shuff-BiseNet road crack segmentation algorithm. Considering the slow speed of detail branch feature extraction, we introduced the basic unit of ShuffleNet to improve the speed of feature extraction and reduce the amount of parameters. Considering the problem of similar pavement cracks and pavement background characteristics, we introduced a variety of attention mechanisms to improve the network’s attention to the pavement crack area and effectively alleviate the impact of the background environment on the segmentation results. Taking into account the problem of pavement crack feature loss caused by using DWConv in the semantic branch, we introduced GSConv to solve the problem of feature loss and improve the receptive field through larger convolution kernel and residual structure. In order to avoid information loss caused by upsampling and downsampling processes and enhance feature expression capabilities, we replaced the pooling stage in the dual-branch feature fusion layer and introduced the DTSConve upsampling module and STDConv downsampling module. On the pavement crack segmentation data set, it achieved accuracy of 97.34%, mPA of 84.34%, IoU (crevice) of 54.81%, and mIoU of 76.03%. Compared with BiseNetv2, mPA, IoU (crevice), and mIoU increased by 4.22%, 4.22%, 2.81%, and 1.36%, respectively.
Similar content being viewed by others
Data availability
In this experiment, CrackForest was selected; the data set was enhanced and added. The datasets during the current study are not publicly available but are available from the corresponding author on reasonable request.
References
Zou, C., Sun, Z., Cai, D., et al.: Crack detection using serrated columnar phased array transducers. Insight Non-Destruct. Test. Cond. Monit. 60(4), 212–219 (2018)
Du, G., Huo, L., Kong, Q., et al.: Damage detection of pipeline multiple cracks using piezoceramic transducers. J. Vibroeng. 18(5), 2828–2838 (2016)
Zima, B., Kdra, R.: Detection and size estimation of crack in plate based on guided wave propagation. Mech. Syst. Signal Process. 142, 106788 (2020)
Chen, Y., Liang, J., Gu, X., et al.: An improved minimal path selection approach with new strategies for pavement crack segmentation. Measurement 184, 109877 (2021)
Amhaz, R., Chambon, S., Idier, J., et al.: Automatic crack detection on two-dimensional pavement images: an algorithm based on minimal path selection. IEEE Trans. Intell. Transp. Syst. 17(10), 2718–2729 (2016)
Kaddah, W., Elbouz, M., Ouerhani, Y., et al.: Optimized minimal path selection (OMPS) method for automatic and unsupervised crack segmentation within two-dimensional pavement images. Vis. Comput. 35(9), 1293–1309 (2019)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Wang, Z., Gao, X., Wu, R., et al.: Fully automatic image segmentation based on FCN and graph cuts. Multimedia Syst. 28(5), 1753–1765 (2022)
Zhang, K., Shi, Y., Hu, C., et al.: Nucleus image segmentation method based on GAN and FCN model. Soft Comput. 26, 7449–7460 (2021)
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18, Springer, pp. 234–241 (2015)
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., et al.: Unet++: a nested u-net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4, Springer, Berlin, pp. 3–11 (2018)
Fu, L., Li, S.: A new semantic segmentation framework based on UNet. Sensors 23(19), 8123 (2023)
Yang, Y., Dasmahapatra, S., Mahmoodi, S.: Ads_unet: a nested UNet for histopathology image segmentation. Expert Syst. Appl. 226, 120128 (2023)
Al-Huda, Z., Peng, B., Algburi, R.N.A., et al.: A hybrid deep learning pavement crack semantic segmentation. Eng. Appl. Artif. Intell. 122, 106142 (2023)
Chen, L.C., Papandreou, G., Kokkinos, I., et al.: Semantic image segmentation with deep convolutional nets and fully connected crfs. (2014) arXiv preprint arXiv:1412.7062
Chen, L.C., Papandreou, G., Kokkinos, I., et al.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)
Chen, L.C., Zhu, Y., Papandreou, G., et al.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 801–818 (2018)
Fu, H., Meng, D., Li, W., et al.: Bridge crack semantic segmentation based on improved deeplabv3+. J. Mar. Sci. Eng. 9(6), 671 (2021)
Yuan, H., Zhu, J., Wang, Q., et al.: An improved deeplab v3+ deep learning network applied to the segmentation of grape leaf black rot spots. Front. Plant Sci. 13, 795410 (2022)
Xie, E., Wang, W., Yu, Z., et al.: Segformer: simple and efficient design for semantic segmentation with transformers. Adv. Neural. Inf. Process. Syst. 34, 12077–12090 (2021)
Cao, H., Wang, Y., Chen, J., et al.: Swin-Unet: Unet-like pure transformer for medical image segmentation. In: European Conference on Computer Vision, Springer, pp. 205–218 (2022)
Liu, H., Yang, J., Miao, X., et al.: Crackformer network for pavement crack segmentation. IEEE Trans. Intell. Transp. Syst. (2023). https://doi.org/10.1109/TITS.2023.3266776
Wang, W., Su, C.: Automatic concrete crack segmentation model based on transformer. Autom. Constr. 139, 104275 (2022)
Zhang, T., Wang, D., Lu, Y.: Ecsnet: an accelerated real-time image segmentation CNN architecture for pavement crack detection. EEE Trans. Intell. Transp. Syst. (2023)
Hou, Y., Liu, S., Cao, D., et al.: A deep learning method for pavement crack identification based on limited field images. IEEE Trans. Intell. Transp. Syst. 23(11), 22156–22165 (2022)
Li, S., Gu, X., Xu, X., et al.: Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm. Constr. Build. Mater. 273, 121949 (2021)
Zhang, T., Wang, D., Mullins, A., et al.: Integrated APC-GAN and AttuNet framework for automated pavement crack pixel-level segmentation: a new solution to small training datasets. IEEE Trans. Intell. Transp. Syst. 24(4), 4474–4481 (2023)
Yu, C., Wang, J., Peng, C., et al.: Bisenet: bilateral segmentation network for real-time semantic segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 325–341 (2018)
Yu, C., Gao, C., Wang, J., et al.: Bisenet v2: bilateral network with guided aggregation for real-time semantic segmentation. Int. J. Comput. Vis. 129, 3051–3068 (2021)
Ma, N., Zhang, X., Zheng, H.T., et al.: Shufflenet v2: practical guidelines for efficient CNN architecture design. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 116–131 (2018)
Li, H., Li, J., Wei, H., et al.: Slim-neck by GSConv: a better design paradigm of detector architectures for autonomous vehicles (2022) arXiv preprint arXiv:2206.02424
Han, K., Wang, Y., Tian, Q., et al.: GhostNet: more features from cheap operations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1580–1589 (2020)
Howard, A., Sandler, M., Chu, G., et al.: Searching for mobilenetv3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1314–1324 (2019)
Ma, N., Zhang, X., Zheng, H.T., et al.: Shufflenet v2: practical guidelines for efficient CNN architecture design. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 116–131 (2018)
Mehta, S., Rastegari, M., Shapiro, L., et al.: Espnetv2: a light-weight, power efficient, and general purpose convolutional neural network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9190–9200 (2019)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Wang, Q., Wu, B., Zhu, P., et al.: Eca-net: efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11534–11542 (2020)
Woo, S., Park, J., Lee, J.Y., et al.: Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)
Liu, M., Zhang, F., Sun, X., et al.: Ca-net: leveraging contextual features for lung cancer prediction. In: Medical Image Computing and Computer Assisted Intervention—MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part V 24, Springer, pp. 23–32 (2021)
Sun, K., Xiao, B., Liu, D., et al.: deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5693–5703 (2019)
Zhao, H., Shi, J., Qi, X., et al.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)
Funding
Not applicable.
Author information
Authors and Affiliations
Contributions
Haiqun mainly completed the writing of the thesis, Bingnan mainly completed the improvement in the model and the ablation experiment and comparison experiment, and Tao mainly completed the comparison of experimental data and the conclusion of the paper
Corresponding author
Ethics declarations
Ethical approval
Not applicable
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, H., Wang, B. & Zhao, T. Shuff-BiseNet: a dual-branch segmentation network for pavement cracks. SIViP 18, 3309–3320 (2024). https://doi.org/10.1007/s11760-023-02993-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-023-02993-2