Skip to main content
SpringerLink
Log in

BLE-Net: boundary learning and enhancement network for polyp segmentation

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Automatic polyp segmentation can improve the accuracy of colonoscopy and plays a crucial role in colorectal cancer prevention. However, existing U-shaped convolutional neural networks fail to satisfactorily localize the boundaries for polyp region, which inevitably degenerates the performance of polyp segmentation. In this article, we propose a boundary learning and enhancement network (BLE-Net) that finely restores edge localization by combining two novel boundary modules. Specifically, a novel boundary learning (BL) module is deployed on the encoder stage to embed edge details into high-level features via a bottom-up fusion way, thereby producing discriminative features with both semantics and boundary information. Moreover, to strengthen the weak responses at fuzzy boundaries, we further design a boundary enhancement (BE) module, in which three cascaded boundary-aware attention blocks progressively endow ambiguous edge cues and rectify preceding maps in a coarse-to-fine fashion. Extensive experimental results on five polyp datasets demonstrate that BLE-Net has excellent segmentation performance and generalization capability, outperforming the state-of-the-arts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Qureshi, K.N., Alhudhaif, A., Ali, M., Qureshi, M.A., Jeon, G.: Self-assessment and deep learning-based coronavirus detection and medical diagnosis systems for healthcare. Multimed. Syst. 1–10, (2021). https://doi.org/10.1007/s00530-021-00839-w

  2. Hao, S., Chen, T., Wang, Y., et al.: Adaptive multi-task dual-structured learning with its application on Alzheimer’s disease Study. ACM Trans. Internet Technol. 21(2), 1–16 (2021)

    Article  Google Scholar 

  3. Ibtehaz, N., Rahman, M.S.: MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw. 121, 74–87 (2020). https://doi.org/10.1016/j.neunet.2019.08.025

    Article  Google Scholar 

  4. Hao, S., Li, G., Wang, L., et al.: Learning-based topological correction for infant cortical surfaces. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 219–227 (2016)

  5. Wang, M., Yao, J., Zhang, G., Guan, B., Wang, X., Zhang, Y.: ParallelNet: multiple backbone network for detection tasks on thigh bone fracture. Multimed. Syst. 27(6), 1091–1100 (2021). https://doi.org/10.1007/s00530-021-00783-9

    Article  Google Scholar 

  6. Sung, H., Ferlay, J., Siegel, R.L., et al.: Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 71(3), 209–249 (2021)

    Article  Google Scholar 

  7. Smith, R.A., Andrews, K.S., Brooks, D., et al.: Cancer screening in the United States, 2018: a review of current American Cancer Society guidelines and current issues in cancer screening. CA Cancer J. Clin. 68(4), 297–316 (2018)

    Article  Google Scholar 

  8. Lui, T.K.L., Hui, C.K.Y., Tsui, V.W.M., et al.: New insights on missed colonic lesions during colonoscopy through artificial intelligence–assisted real-time detection (with video). Gastrointest. Endosc. 93(1), 193–200 (2021)

    Article  Google Scholar 

  9. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention (MICCAI), pp. 234–241 (2015)

  10. Liu, Z., Han, K., Wang, Z., Zhang, J., Song, Y., Yao, X., Yuan, D., Sheng, V.S.: Automatic liver segmentation from abdominal CT volumes using improved convolution neural networks. Multimed. Syst. 27(1), 111–124 (2021). https://doi.org/10.1007/s00530-020-00709-x

    Article  Google Scholar 

  11. Feng, S., Zhao, H., Shi, F., et al.: CPFNet: context pyramid fusion network for medical image segmentation. IEEE Trans. Med. Imaging 39(10), 3008–3018 (2020). https://doi.org/10.1109/TMI.2020.2983721

    Article  Google Scholar 

  12. Araújo, R.L., de Araújo, F.H., Silva, R.R.: Automatic segmentation of melanoma skin cancer using transfer learning and fine-tuning. Multimed. Syst. 1–12 (2021). https://doi.org/10.1007/s00530-021-00840-3

  13. Li, X., Jiang, Y., Li, M., Yin, S.: Lightweight attention convolutional neural network for retinal vessel image segmentation. IEEE Trans. Ind. Inf. 17(3), 1958–1967 (2020). https://doi.org/10.1109/TII.2020.2993842

    Article  Google Scholar 

  14. Yu, J., Pan, H., Yin, Q., Bian, X., Cui, Q.: Fully convolutional densenets for polyp segmentation in colonoscopy. In: 2019 IEEE 35th International Conference on Data Engineering Workshops (ICDEW), pp. 306–311 (2019). https://doi.org/10.1109/ICDEW.2019.00010

  15. Guo, X., Zhang, N., Guo, J., Zhang, H., Hao, Y., Hang, J.: Automated polyp segmentation for colonoscopy images: a method based on convolutional neural networks and ensemble learning. Med. Phys. 46(12), 5666–5676 (2019)

    Article  Google Scholar 

  16. Mahmud, T., Paul, B., Fattah, S.A.: PolypSegNet: a modified encoder-decoder architecture for automated polyp segmentation from colonoscopy images. Comput. Biol. Med. 128, 104119 (2021). https://doi.org/10.1016/j.compbiomed.2020.104119

    Article  Google Scholar 

  17. Zhou, Z., Siddiquee, M., Tajbakhsh, N., Liang, J.: Unet++: a nested u-net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 3–11 (2018)

  18. Fan, D.P., Ji, G.P., Zhou, T., Chen, G., Fu, H., Shen, J., Shao, L.: Pranet: parallel reverse attention network for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 263–273 (2020)

  19. Fang, Y., Chen, C., Yuan, Y., Tong, K.Y.: Selective feature aggregation network with area-boundary constraints for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 302–310 (2019)

  20. Murugesan, B., Sarveswaran, K., Shankaranarayana, S.M., Ram, K., Joseph, J., Sivaprakasam, M.: Psi-Net: shape and boundary aware joint multi-task deep network for medical image segmentation. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 7223–7226 (2019). https://doi.org/10.1109/EMBC.2019.8857339

  21. Fang, Y., Zhu, D., Yao, J., Yuan, Y., Tongy, K.Y.: ABC-Net: area-boundary constraint network with dynamical feature selection for colorectal polyp segmentation. IEEE Sens. J. 21(10), 11799–11809 (2020)

    Article  Google Scholar 

  22. Murugesan, B., Sarveswaran, K., Shankaranarayana, S.M., Ram, K.: Joint shape learning and segmentation for medical images using a minimalistic deep network (2019) . arXiv:1901.08824

  23. Guo, Y. B., Matuszewski, B.: Giana polyp segmentation with fully convolutional dilation neural networks. In: Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, pp. 632–641 (2019). https://doi.org/10.5220/0007698806320641

  24. Jha, D., Smedsrud, P.H., Riegler, M.A., Johansen, D., De Lange, T., Halvorsen, P., Johansen, H.D.: Resunet++: an advanced architecture for medical image segmentation. In: IEEE International Symposium on Multimedia (ISM), pp. 225–230 (2019). https://doi.org/10.1109/ISM46123.2019.00049

  25. Zhong, J., Wang, W., Wu, H., Wen, Z., Qin, J.: PolypSeg: an efficient context-aware network for polyp segmentation from colonoscopy videos. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 285–294 (2020)

  26. Zhang, R., Li, G., Li, Z., Cui, S., Qian, D., Yu, Y.: Adaptive context selection for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 253–262 (2020)

  27. Jia, X., Mai, X., Cui, Y., Yuan, Y., Meng, Q.H.: Automatic polyp recognition in colonoscopy images using deep learning and two-stage pyramidal feature prediction. IEEE Trans. Autom. Sci. Eng. 17(3), 1570–1584 (2020). https://doi.org/10.1109/TASE.2020.2964827

    Article  Google Scholar 

  28. Sun, X., Zhang, P., Wang, D., Cao, Y., Liu, B.: Colorectal polyp segmentation by u-net with dilation convolution. In: 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), pp. 851–858 (2019). https://doi.org/10.1109/ICMLA.2019.00148

  29. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7132–7141 (2018)

  30. Lin, D., Li, Y., New, T.L., Dong, S., Oo, Z.M.: RefineU-Net: improved U-Net with progressive global feedbacks and residual attention guided local refinement for medical image segmentation. Pattern Recogn. Lett. 138, 267–275 (2020). https://doi.org/10.1016/j.patrec.2020.07.013

    Article  Google Scholar 

  31. Feng, R., Lei, B., Wang, W., Chen, T., Chen, J., Chen, D.Z., Wu, J.: SSN: a stair-shape network for real-time polyp segmentation in colonoscopy images. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 225–229 (2020). https://doi.org/10.1109/ISBI45749.2020.9098492

  32. Luo, Y., Zhou, X., Li, K., Yao, Z.: Accurate gastric ulcer and intestinal polyp segmentation using attention network and multiscale Information Integration. In: 2020 International Conference on Artificial Intelligence and Computer Engineering (ICAICE), pp. 542–546 (2020)

  33. Zhang, D., Zhang, H., Tang, J., Wang, M., Hua, X., Sun, Q.: Feature pyramid transformer. In: European Conference on Computer Vision (ECCV), pp. 323–339 (2020)

  34. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1492–1500 (2017). https://doi.org/10.1109/CVPR.2017.634

  35. Wu, Z., Su, L., Huang, Q.: Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3907–3916 (2019)

  36. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: Learning a discriminative feature network for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1857–1866 (2018)

  37. Wei, J., Wang, S., Huang, Q.: F\(^{3}\)Net: fusion, feedback and focus for salient object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), vol. 34(07), pp. 12321–12328 (2020)

  38. Bernal, J., Sánchez, F.J., Fernández-Esparrach, G., Gil, D., Rodríguez, C., Vilariño, F.: WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput. Med. Imaging Graph. 43, 99–111 (2015). https://doi.org/10.1016/j.compmedimag.2015.02.007

    Article  Google Scholar 

  39. Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., de Lange, T., Johansen, D., Johansen, H.D.: Kvasir-seg: a segmented polyp dataset. In: International Conference on Multimedia Modeling, pp. 451–462 (2020)

  40. Tajbakhsh, N., Gurudu, S.R., Liang, J.: Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans. Med. Imaging 35(2), 630–644 (2015). https://doi.org/10.1109/TMI.2015.2487997

    Article  Google Scholar 

  41. Silva, J., Histace, A., Romain, O., Dray, X., Granado, B.: Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int. J. Comput. Assist. Radiol. Surg. 9(2), 283–293 (2014). https://doi.org/10.1007/s11548-013-0926-3

    Article  Google Scholar 

  42. Vázquez, D., Bernal, J., Sánchez, F.J., et al.: A benchmark for endoluminal scene segmentation of colonoscopy images. J. Healthc. Eng. 2017, (2017). https://doi.org/10.1155/2017/4037190

  43. Margolin, R., Zelnik-Manor, L., Tal, A.: How to evaluate foreground maps? In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 248–255 (2014). https://doi.org/10.1109/CVPR.2014.39

  44. Fan, D. P., Gong, C., Cao, Y., Ren, B., Cheng, M. M., Borji, A.: Enhanced-alignment measure for binary foreground map evaluation. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, pp. 698–704 (2018)

  45. Fan, D.P., Cheng, M.M., Liu, Y., Li, T., Borji, A.: Structure-measure: a new way to evaluate foreground maps. In: Proceedings of the IEEE International Conference on Computer Vision (CVPR), pp. 4548–4557 (2017). https://doi.org/10.1109/ICCV.2017.487

  46. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015). https://doi.org/10.1109/CVPR.2015.7298965

  47. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017). https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  48. Oktay, O., Schlemper, J., Folgoc, L.L., et al.: Attention u-net: learning where to look for the pancreas (2018). arXiv:1804.03999

Download references

Acknowledgements

This research is supported by the National Key Research and Development Program of China (2018 YFB0804202, 2018YFB0804203), the Regional Joint Fund of NSFC (U19A2057), the National Natural Science Foundation of China (61876070), the Jilin Province Science and Technology Development Plan Project (20190303134SF), the Science and Technology Planning Project of Inner Mongolia (2020GG0130), the Natural Science Foundation of Inner Mongolia (2020MS04007), and the Ph.D. Foundation of Hulunbuir University (2020BS11).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Na Ta & Haipeng Chen

  2. College of Computer, Hulunbuir University, Hulunbuir, 021008, China

    Na Ta

  3. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Na Ta & Haipeng Chen

  4. Public Computer Education and Research Center, Jilin University, Changchun, 130012, China

    Yingda Lyu

  5. School of Physics and Electronic Information, Hulunbuir University, Hulunbuir, 021008, China

    Taosuo Wu

Authors
  1. Na Ta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haipeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yingda Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Taosuo Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingda Lyu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ta, N., Chen, H., Lyu, Y. et al. BLE-Net: boundary learning and enhancement network for polyp segmentation. Multimedia Systems 29, 3041–3054 (2023). https://doi.org/10.1007/s00530-022-00900-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-00900-2

Keywords

Search

Navigation