Skip to main content
SpringerLink
Log in

Multi-decoding Network with Attention Learning for Edge Detection

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In the past few years, edge detection models based on convolutional neural networks have made remarkable progress. These models consist of the encoding network and the decoding network. The classification networks (e.g. VGG16) are generally used as the encoding network and the researchers mainly focus on the design of the decoding network. Because natural images contain many edges with different scales, how to make full use of rich and hierarchical convolutional features for edge detection is pivotal. We propose a decoding network that effectively integrates multi-level features from all convolutional layers, named multi-decoding network. Firstly, the side outputs from VGG16 are divided into shallow-, mid-, and deep-level edge features. These edge features are processed separately by multiple independent decoding networks composed of several refinement blocks. Subsequently, to improve the ability of edge fusion, we design a multi-level top-down architecture and a multi-scale attention module, which utilizes the attention mechanism to gradually refine edges. Ultimately, three types of predictions are averaged to produce the final prediction. We evaluate our method on the BSDS500, NYUDv2, and Multicue datasets. Experimental results demonstrate that our model is superior to multiple state-of-the-art edge detection models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Li C, Xia W, Yan Y et al (2021) Segmenting objects in day and night: edge-conditioned cnn for thermal image semantic segmentation. IEEE Trans Neural Netw Learn Syst 32(7):3069–3082. https://doi.org/10.1109/TNNLS.2020.3009373

    Article  Google Scholar 

  2. Zhao L, Zhao Q, Liu H et al (2017) Structural sparse representation-based semi-supervised learning and edge detection proposal for visual tracking. Vis Comput 33(9):1169–1184. https://doi.org/10.1007/s00371-016-1279-z

    Article  Google Scholar 

  3. Zhao JX, Liu JJ, Fan DP et al (2019) EGNet: edge guidance network for salient object detection. In: IEEE international conference on computer vision, Seoul, Korea (South), pp 8779–8788. https://doi.org/10.1109/ICCV.2019.00887

  4. Rampun A, López-Linares K, Morrow PJ (2019) Breast pectoral muscle segmentation in mammograms using a modified holistically-nested edge detection network. Med Image Anal 57:1–17. https://doi.org/10.1016/j.media.2019.06.007

    Article  Google Scholar 

  5. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell PAMI 8(6):679–698. https://doi.org/10.1109/TPAMI.1986.4767851

    Article  Google Scholar 

  6. Duda RO, Hart PE (1973) Pattern classification and scene analysis. Wiley, New York

    MATH  Google Scholar 

  7. Grigorescu C, Petkov N, Westenberg MA (2003) Contour detection based on nonclassical receptive field inhibition. IEEE Trans Image Process 12(7):729–739. https://doi.org/10.1109/TIP.2003.814250

    Article  Google Scholar 

  8. Yang KF, Gao SB, Guo CF et al (2015) Boundary detection using double-opponency and spatial sparseness constraint. IEEE Trans Image Process 24(8):2565–2578. https://doi.org/10.1109/TIP.2015.2425538

    Article  MathSciNet  MATH  Google Scholar 

  9. Akbarinia A, Parraga CA (2018) Feedback and surround modulated boundary detection. Int J Comput Vis 126(12):1367–1380. https://doi.org/10.1007/s11263-017-1035-5

    Article  Google Scholar 

  10. Arbelaez P, Maire M, Fowlkes C et al (2010) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916. https://doi.org/10.1109/TPAMI.2010.161

    Article  Google Scholar 

  11. Martin DR, Fowlkes CC, Malik J (2004) Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans Pattern Anal Mach Intell 26(5):530–549. https://doi.org/10.1109/TPAMI.2004.1273918

    Article  Google Scholar 

  12. Hallman S, Fowlkes CC (2015) Oriented edge forests for boundary detection. In: IEEE conference on computer vision and pattern recognition, Boston, MA, USA, pp 1732–1740. https://doi.org/10.1109/CVPR.2015.7298782

  13. Xie S, Tu Z (2015) Holistically-nested edge detection. In: IEEE international conference on computer vision, Santiago, Chile, pp 1395–1403. https://doi.org/10.1109/ICCV.2015.164

  14. Liu Y, Cheng MM, Hu X et al (2018) Richer convolutional features for edge detection. IEEE Trans Pattern Anal Mach Intell 41(8):1939–1946. https://doi.org/10.1109/TPAMI.2018.2878849

    Article  Google Scholar 

  15. Wang Y, Zhao X, Li Y et al (2018) Deep crisp boundaries: from boundaries to higher-level tasks. IEEE Trans Image Process 28(3):1285–1298. https://doi.org/10.1109/TIP.2018.2874279

    Article  MathSciNet  MATH  Google Scholar 

  16. Yang H, Li Y, Yan X et al (2019) ContourGAN: image contour detection with generative adversarial network. Knowl Based Syst 164:21–28. https://doi.org/10.1016/j.knosys.2018.09.033

    Article  Google Scholar 

  17. Zhang R, You M (2021) Fast contour detection with supervised attention learning. J Real Time Image Process 18(3):647–657. https://doi.org/10.1007/s11554-020-00980-1

    Article  Google Scholar 

  18. Cao YJ, Lin C, Li YJ et al (2020) Learning crisp boundaries using deep refinement network and adaptive weighting loss. IEEE Trans Multimed 23:761–771. https://doi.org/10.1109/TMM.2020.2987685

    Article  Google Scholar 

  19. Silberman N, Hoiem D, Kohli P et al (2012) Indoor segmentation and support inference from RGBD images. In: European conference on computer vision, Florence, Italy, pp 746–760. https://doi.org/10.1007/978-3-642-33715-4_54

  20. Mély DA, Kim J, McGill M et al (2016) A systematic comparison between visual cues for boundary detection. Vis Res 120:93–107. https://doi.org/10.1016/j.visres.2015.11.007

    Article  Google Scholar 

  21. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations, San Diego, CA, USA, pp 1–14. https://arxiv.org/abs/1409.1556

  22. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  23. Lin T, Dollár P, Girshick R et al (2017) Feature pyramid networks for object detection. In: IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, pp 936–944. https://doi.org/10.1109/CVPR.2017.106

  24. Chen L, Papandreou G, Kokkinos I et al (2017) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  25. Sandler M, Howard A, Zhu M et al (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, pp 4510–4520. https://doi.org/10.1109/CVPR.2018.00474

  26. Woo S, Park J, Lee JY et al (2018) CBAM: convolutional block attention module. In: European conference on computer vision, Munich, Germany, pp 3–19. https://doi.org/10.1007/978-3-030-01234-2_1

  27. Hendrycks D, Gimpel K (2016) Gaussian error linear units (gelus). Preprint https://arxiv.org/abs/1606.08415

  28. Mottaghi R, Chen X, Liu X et al (2014) The role of context for object detection and semantic segmentation in the wild. In: IEEE conference on computer vision and pattern recognition, Columbus, OH, USA, pp 891–898. https://doi.org/10.1109/CVPR.2014.119

  29. Gupta S, Girshick R, Arbeláez P et al (2014) Learning rich features from RGB-D images for object detection and segmentation. In: European conference on computer vision, Zurich, Switzerland, pp 345–360. https://doi.org/10.1007/978-3-319-10584-0_23

  30. Deng R, Shen C, Liu S et al (2018) Learning to predict crisp boundaries. In: European conference on computer vision, Zurich, Switzerland, pp 562–578. https://doi.org/10.1007/978-3-030-01231-1_35

  31. He J, Zhang S, Yang M et al (2019) Bi-directional cascade network for perceptual edge detection. In: IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp 3828–3837. https://doi.org/10.1109/CVPR.2019.00395

  32. Shen W, Wang X, Wang Y et al (2015) Deepcontour: a deep convolutional feature learned by positive-sharing loss for contour detection. In: IEEE conference on computer vision and pattern recognition, Boston, MA, USA, pp 3982–3991. https://doi.org/10.1109/CVPR.2015.7299024

  33. Bertasius G, Shi J, Torresani L (2015) Deepedge: a multi-scale bifurcated deep network for top-down contour detection. In: IEEE conference on computer vision and pattern recognition, Boston, MA, USA, pp 4380–4389. https://doi.org/10.1109/CVPR.2015.7299067

  34. Maninis KK, Pont-Tuset J, Arbeláez P et al (2017) Convolutional oriented boundaries: from image segmentation to high-level tasks. IEEE Trans Pattern Anal Mach Intell 27:1676–1688. https://doi.org/10.1109/TPAMI.2017.2700300

    Article  Google Scholar 

  35. Yang JM, Price B, Cohen S et al (2016) Object contour detection with a fully convolutional encoder-decoder network. In: IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, pp 193–202. https://doi.org/10.1109/CVPR.2016.28

  36. Kelm AP, Rao VS, Zölzer U (2019) Object contour and edge detection with refinecontournet. In: 18th Computer analysis of images and patterns, Salerno, Italy, pp 246–258. https://doi.org/10.1007/978-3-030-29888-3_20

  37. Dollár P, Zitnick CL (2015) Fast edge detection using structured forests. IEEE Trans Pattern Anal Mach Intell 37(8):1558–1570. https://doi.org/10.1109/TPAMI.2014.2377715

    Article  Google Scholar 

  38. Gupta S, Arbelaez P, Malik J (2013) Perceptual organization and recognition of indoor scenes from RGB-D images. In: IEEE conference on computer vision and pattern recognition, Portland, OR, USA, pp 564–571. https://doi.org/10.1109/CVPR.2013.79

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 61866002), Guangxi Natural Science Foundation (Grant Nos. 2020GXNSFDA297006, 2018GXNSFAA138122, 2015GXNSFAA139293), and Innovation Project of Guangxi Graduate Education (Grant No. YCSW2021301).

Author information

Authors and Affiliations

  1. School of Electrical, Electronic and Computer Science, Guangxi University of Science and Technology, Liuzhou, 545616, China

    Xiao Zhang & Chuan Lin

Authors
  1. Xiao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chuan Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chuan Lin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Lin, C. Multi-decoding Network with Attention Learning for Edge Detection. Neural Process Lett 55, 4889–4906 (2023). https://doi.org/10.1007/s11063-022-11070-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-11070-7

Keywords

Search

Navigation