Skip to main content
SpringerLink
Log in

Developing a feature decoder network with low-to-high hierarchies to improve edge detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Low-to-high hierarchical convolutional features can significantly improve edge detection. This paper proposes a feature decoder-based algorithm that employs a Feature Decoder Network (FDN) to extract more information within limited Convolutional Neural Network (CNN) features. Previous studies applied convolutional elements by weight fusion, but we measure a feature decoder as a pyramid by qualifying convolutional layers. The feature decoder fuses CNN features of adjacent layers to judge the edge and non-edge pixels, which can learn the relationship and distinction between low-level edge hierarchies and high-level semantic hierarchies. Furthermore, we use Gaussian blur labels to train the network to optimize network convergence and training. From the experimental results, our proposed algorithm performs better on the BSDS500 (average accuracy (AP) of 0.865) and NYUD (OIS F-measure of 0.775) datasets compared to the state-of-the-art algorithms, including RCF.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  2. Arbelaez P, Pont-Tuset J, Barron J, Marques F, Malik J (2014) Multiscale combinatorial grouping. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 328–335

  3. Bertasius G, Shi J, Torresani L (2015) DeepEdge: a multiscale bifurcated deep network for top-down contour detection. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 4380–4389

  4. Bertasius G, Shi J, Torresani L (2015) High-for-low and lowfor-high: efficient boundary detection from deep object features and its applications to high-level vision. In: Proc IEEE Int Conf Comput Vis pp. 504–512

  5. Canny J (1987) A computational approach to edge detection. In: Readings Comput Vis pp. 184–203

  6. Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille A (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. CoRR IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  7. Cheng M, Liu Y, Hou Q, Bian J, Torr P, Hu S, Tu Z (2016) HFS: hierarchical feature selection for efficient image segmentation. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 867c882

  8. Choi Y, Choi M, Kim M, Ha J, Kim S, Choo J (2018) Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 8789–8797

  9. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  10. Dollar P, Zitnick C (2014) Fast edge detection using structured forests. IEEE Trans Pattern Anal Mach Intell 37(8):1558–1570

    Article  Google Scholar 

  11. Dollar P, Tu Z, Belongie S (2006) Supervised learning of edges and object boundaries. In: Proc IEEE Conf Comput Vis Pattern Recognitpp. 1964–1971

  12. Duda R, Hart P (1974) Pattern classification and scene analysis. IEEE Trans Automat Contr 19(4):462–463

    Article  Google Scholar 

  13. Fang T, Fan Y, Wu W (2020) Salient contour detection on the basis of the mechanism of bilateral asymmetric receptive fields. Signal Image Video Process. https://doi.org/10.1007/s11760-020-01689-1

  14. Felzenszwalb P, Huttenlocher D (2004) Efficient graphbased image segmentation. Int J Comput Vis 59(2):167–181

    Article  Google Scholar 

  15. Ferrari V, Fevrier L, Jurie F, Schmid C (2007) Groups of adjacent contour segments for object detection. IEEE Trans Pattern Anal Mach Intell 30(1):36–51

    Article  Google Scholar 

  16. Ganin Y, Lempitsky V (2014) N4-fields: neural network nearest neighbor fields for image transforms. In: Asian Conf Comput Vis pp. 536–551

  17. Gupta S, Arbelaez P, Malik J (2013) Perceptual organization and recognition of indoor scenes from RGB-d images. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 564–571

  18. Gupta S, Girshick R, Arbelaez P, Malik J (2014) Learning rich features from RGB-d images for object detection and segmentation. In: Euro Conf Comput Vis pp. 345–360

  19. Hallman S, Fowlkes C (2015) Oriented edge forests for boundary detection. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 1732–1740

  20. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 770–778

  21. He J, Zhang S, Yang M, Shan Y, Huang T (2019) Bi-directional cascade network for perceptual edge detection. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 3828–3837

  22. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proc ACM Int Conf Multi pp. 675–678

  23. Lim J, Zitnick C, Dollar P (2013) Sketch tokens: a learned mid-level representation for contour and object detection. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 3158–3165

  24. Liu Y, Cheng M, Hu X, Wang K, Bai X (2017) Richer convolutional features for edge detection. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 3000–3009

  25. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In Proc IEEE Conf Comput Vis Pattern Recognit pp. 3431–3440

  26. Maninis K, Pont-Tuset J, Arbeláez P, Van-Gool L (2016) Convolutional oriented boundaries. In: Euro Conf Comput Vis pp. 580–596

  27. Mottaghi R, Chen X, Liu X, Cho N, Lee S, Fidler S, Urtasun R, Yuille A (2014) The role of context for object detection and semantic segmentation in the wild. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 891–898

  28. Poma X, Riba E, Sappa A (2020) Dense extreme inception network: towards a robust cnn model for edge detection. In: IEEE Winter Conf Appl Comput Vis pp. 1923–1932

  29. Ren X, Bo L (2012) Discriminatively trained sparse code gradients for contour detection. Int Conf Neural Informa Process Syst pp. 584–592.

  30. Ren Z, Shakhnarovich G (2013) Image segmentation by cascaded region agglomeration. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 2011–2018

  31. Shen W, Wang X, Wang Y, Bai X, Zhang Z (2015) DeepContour: a deep convolutional feature learned by positivesharing loss for contour detection. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 3982–3991

  32. Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from RGB-d images. In: Euro Conf Comput Vis pp. 746–760

  33. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  34. Sobel I (1972) Camera models and machine perception. Ph.d Thesis, Stanford University, Stanford

  35. Tang P, Wang H, Kwong S (2017) G-MS2F: GoogLeNet based multi-stage feature fusion of deep CNN for scene recognition. Neurocomputing 225:188–197

    Article  Google Scholar 

  36. Wang X, Wu C, Xiang K, Xiang S, Chen W (2018) An experimental comparison of superpixels detection methods for contour detection. Mach Vis Appl 29(4):677–687

    Article  Google Scholar 

  37. Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proc IEEE Int Conf Comput Vis pp. 1395–1403

  38. Xu D, Ouyang W, Alameda-Pineda X, Ricci E, Wang X, Sebe N (2017) Learning deep structured multi-scale features using attention-gated crfs for contour prediction. Int Conf Neural Informa Process Syst pp. 3961–3970

  39. Yang J, Yang M (2016) Top-down visual saliency via joint CRF and dictionary learning. IEEE Trans Pattern Anal Mach Intell 39(3):576–588

    Article  Google Scholar 

  40. Yang K, Li C, Li Y (2014) Multifeature-based surround inhibition improves contour detection in natural images. IEEE Trans Image Process 23(12):5020–5032

    Article  MathSciNet  Google Scholar 

  41. Yang J, Price B, Cohen S, Lee H, Yang M (2016) Object contour detection with a fully convolutional encoder-decoder network. In: Proc IEEE Conf Comput Vis Pattern Recognit pp. 193–202

Download references

Acknowledgments

The work was supported in part by the National Natural Science Foundation of China (61501154). The authors would like to thank Yun Liu for his kind and help in the writing process.

Author information

Authors and Affiliations

  1. Laboratory of Pattern Recognition and Image Processing Hangzhou Dianzi University, Hangzhou, 310018, China

    Tao Fang, Mingqi Zhang, Yingle Fan, Wei Wu, Haitao Gan & Qingshan She

Authors
  1. Tao Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingqi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yingle Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haitao Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qingshan She
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingle Fan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, T., Zhang, M., Fan, Y. et al. Developing a feature decoder network with low-to-high hierarchies to improve edge detection. Multimed Tools Appl 80, 1611–1624 (2021). https://doi.org/10.1007/s11042-020-09800-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09800-x

Keywords

Search

Navigation