Abstract
The presence of road cracks significantly impacts both traffic safety and road maintenance. Therefore, accurate detection of road cracks plays a crucial role in road maintenance and management. This study focused on addressing the critical challenge of accurate road crack detection in images through the development of a novel network architecture (PMENet) based on deep learning semantic segmentation algorithms. The proposed PMENet algorithm combined parallel contextual squeeze and excitation(PCSE), multiscale feature fusion(MFF) and double residual efficient attention(DREA) modules based on the UNet structure, which improves the ability to extract global local information and crack boundary information from the model. We thoroughly evaluate the performance of the PMENet algorithm on the publicly available Crack500 dataset by the ablation experiment. We compared our method with existing approaches, demonstrating its superiority in terms of accuracy, robustness, and generalization ability. The PMENet algorithm achieves an accuracy of 96.14%, an F1-score of 84.22%, and an IoU of 62.00%. The proposed model achieves a significant improvement of 5.9% in F1-score and 5.49% in IoU compared to the UNet model.
Similar content being viewed by others
References
Jung, J.-Y., Yoon, H.-J., Cho, H.-W.: A study on crack depth measurement in steel structures using image-based intensity differences. Adv. Civ. Eng. (2018). https://doi.org/10.1155/2018/7530943
Zhu, Z., German, S., Brilakis, I.: Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation. Autom. Constr. 20(7), 874–883 (2011). https://doi.org/10.1016/j.autcon.2011.03.004
Er-Sen, L., Shu-Long, Z., Bao-shan, Z., Yong, Z., Chao-gui, X., Li-hua, S.: An adaptive edge-detection method based on the canny operator. In: 2009 International Conference on Environmental Science and Information Application Technology, vol. 1, pp. 465–469. https://doi.org/10.1109/ESIAT.2009.49
Kanopoulos, N., Vasanthavada, N., Baker, R.L.: Design of an image edge detection filter using the sobel operator. IEEE J. Solid-State Circ. 23(2), 358–367 (1988). https://doi.org/10.1109/4.996
Manjunath, B.S., Ma, W.-Y.: Texture features for browsing and retrieval of image data. IEEE Trans. Pattern Anal. Mach. Intell. 18(8), 837–842 (1996). https://doi.org/10.1109/34.531803
Sholevar, N., Golroo, A., Esfahani, S.R.: Machine learning techniques for pavement condition evaluation. Autom. Constr. 136, 104190 (2022). https://doi.org/10.1016/j.autcon.2022.104190
Wang, W., Wang, M., Li, H., Zhao, H., Wang, K., He, C., Wang, J., Zheng, S., Chen, J.: Pavement crack image acquisition methods and crack extraction algorithms: a review. J. Traffic Transp. Eng. 6(6), 535–556 (2019). https://doi.org/10.1016/j.jtte.2019.10.001
Gavilan, M., Balcones, D., Marcos, O., Llorca, D.F., Sotelo, M.A., Parra, I., Ocana, M., Aliseda, P., Yarza, P., Amirola, A.: Adaptive road crack detection system by pavement classification. Sensors 11(10), 9628–9657 (2011). https://doi.org/10.3390/s111009628
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539
Cha, Y.-J., Choi, W., Buyukozturk, O.: Deep learning-based crack damage detection using convolutional neural networks. Comput. Aided Civ. Infrastruct. Eng. 32(5), 361–378 (2017). https://doi.org/10.1111/mice.12263
Liu, C., Zhu, C., Xia, X., Zhao, J., Long, H.: FFEDN: Feature fusion encoder decoder network for crack detection. IEEE Transactions on Intelligent Transportation Systems 23(9), 15546–15557 https://doi.org/10.1109/TITS.2022.3141827
Zou, Q., Zhang, Z., Li, Q., Qi, X., Wang, Q., Wang, S.: DeepCrack: learning hierarchical convolutional features for crack detection. IEEE Transactions on Image Processing 28(3), 1498–1512 https://doi.org/10.1109/TIP.2018.2878966
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer
Wan, H., Gao, L., Su, M., Sun, Q., Huang, L.: Attention-based convolutional neural network for pavement crack detection. Adv. Mater. Sci. Eng. (2021). https://doi.org/10.1155/2021/5520515
Lau, S. L., Chong, E. K., Yang, X., Wang, X: Automated pavement crack segmentation using u-net-based convolutional neural network. IEEE Transactions on Image Processing 8, 114892–114899 https://doi.org/10.1109/ACCESS.2020.3003638
Han, C., Ma, T., Huyan, J., Huang, X., Zhang, Y.: CrackW-net: a novel pavement crack image segmentation convolutional neural network. IEEE Trans. Intell. Transp. Syst. 23(11), 22135–22144 (2021). https://doi.org/10.1109/TITS.2021.3095507
Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., Kainz, B., et al.: Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018)
Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018). arXiv:1709.01507v4
Q., W., B., W., P., Z., P., L., W., Z., Q., H.: ECA-Net: Efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11531–11539. https://doi.org/10.1109/CVPR42600.2020.01155
Roy, A.G., Navab, N., Wachinger, C.: Concurrent spatial and channel squeeze & excitation in fully convolutional networks. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I, pp.421–429 (2018). Springer. arXiv:1803.02579v2
F., M., N., N., S., A.A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. https://doi.org/10.1109/3DV.2016.79
Yang, F., Zhang, L., Yu, S., Prokhorov, D., Mei, X., Ling, H.: Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Trans. Intell. Transp. Syst. 21(4), 1525–1535 (2019). https://doi.org/10.1109/TITS.2019.2910595
Shi, Y., Cui, L., Qi, Z., Meng, F., Chen, Z.: Automatic road crack detection using random structured forests. IEEE Trans. Intell. Transp. Syst. 17(12), 3434–3445 (2016). https://doi.org/10.1109/TITS.2016.2552248
Zhao, H., Qi, X., Shen, X., Shi, J., Jia, J.: ICNet for real-time semantic segmentation on high-resolution images. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 405–420 (2018). arXiv:1704.08545v2
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2881–2890 (2017). arXiv:1612.01105v2
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). arXiv:1512.03385v1
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: UNet++: A nested u-net architecture for medical image segmentation, pp. 3–11 (2018). Springer. arXiv:1807.10165v1
Yu, G., Dong, J., Wang, Y., Zhou, X.: RUC-Net: a residual-unet-based convolutional neural network for pixel-level pavement crack segmentation. Sensors 23(1), 53 (2022)
Shi, P., Zhu, F., Xin, Y., Shao, S.: U2CrackNet: a deeper architecture with two-level nested u-structure for pavement crack detection. Struct. Health Monitor. 22(4), 2910–2921 (2023). https://doi.org/10.1177/14759217221140976
Kato, S., Hotta, K.: Mse loss with outlying label for imbalanced classification. arXiv preprint arXiv:2107.02393 (2021)
Mannor, S., Peleg, D., Rubinstein, R.: The cross entropy method for classification. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 561–568 (2005). https://doi.org/10.1145/1102351.1102422
Phan, T.H., Yamamoto, K.: Resolving class imbalance in object detection with weighted cross entropy losses. arXiv preprint arXiv:2006.01413 (2020)
Funding
This work was supported in part by the National Natural Science Foundation of China (Grant no. 51805124) and in part by the Zhejiang Provincial Natural Science Foundation of China (Grant no. LZY22E050001 and LZY24E050001)
Author information
Authors and Affiliations
Contributions
BW and CD contributed to the study design, implementation, results analysis and preparation of the final draft. JL and XJ contributed to the design and preparation of the dataset. JZ and GJ contributed to the graph. All authors reviewed the manuscript.
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare no Conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, B., Dai, C., Li, J. et al. PMENet: a parallel UNet based on the fusion of multiple attention mechanisms for road crack segmentation. SIViP 18 (Suppl 1), 757–769 (2024). https://doi.org/10.1007/s11760-024-03190-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-024-03190-5