Skip to main content
SpringerLink
Log in

Feature enhancement: predict more detailed and crisper edges

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

CNN-based methods have improved the performance of edge detection in recent years. While a common issue with the most recent methods is that there is a thickness problem in predicting the edge map, and when objects are small or have dense edges, the predicted edge lines are blurred. We know that multi-pooling reduces the resolution of features, and using a balanced cross-entropy loss function will also make the predicted edges thicker. In this paper, we propose a multi-scale feature hybrid network for edge detection to improve edge resolution. Multiple long and short residual connections are set to establish various information propagation pathways. We construct a feature enhancement unit in the up-sampling path of the network to obtain multi-scale features and fuse higher-resolution features. We demonstrate that residual connections in the network can overcome the class-imbalance training problem in edge samples. Moreover, we introduce a new biased cross-entropy loss function to accomplish the training of our network better, which adjusts the weights according to the ratio of edges and non-edges pixels. Compared to other methods, our network can predict clearer and sharper edges with more details. Evaluate the network on BSDS500 and NYUDv2, our method achieves ODS F-measure of 0.832 on the BSDS500 dataset and 0.768 on the NYUD dataset, best than current state-of-the-art results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011). https://doi.org/10.1109/TPAMI.2010.161

    Article  Google Scholar 

  2. Chen, L., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018). https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  3. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: 12th European Conference on Computer Vision, vol. 7576, no. 5, pp. 746–760 (2012)

  4. Xie, S., Tu, Z.: Holistically-nested edge detection. In: IEEE International Conference on Computer Vision, Santiago, pp. 1395–1403 (2015). https://doi.org/10.1109/ICCV.2015.164

  5. Liu, Y., Cheng, M., Hu, X., Wang, K., Bai, X.: Richer convolutional features for edge detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, pp. 5872–5881 (2017). https://doi.org/10.1109/CVPR.2017.622

  6. Wang, Y., Zhao, X., Huang, K.: Deep crisp boundaries. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, pp. 1724–1732 (2017). https://doi.org/10.1109/CVPR.2017.187

  7. Deng, R., Shen, C., Liu, S., Wang, H., Liu, X.: Learning to predict crisp boundaries. In: Computer Vision—ECCV 2018, vol. 11210. Springer, Cham. https://doi.org/10.1007/978-3-030-01231-1_35

  8. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  9. He, J., Zhang, S., Yang, M., et al.: Bi-directional cascade network for perceptual edge detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 3823–3832 (2019). https://doi.org/10.1109/CVPR.2019.00395

  10. He, K., Zhang, X., Ren, S, et al.: Identity mappings in deep residual networks. arXiv preprint arXiv:1603.05027 (2016)

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 770–778 (2016)

  12. Szegedy, C., Ioffe, S., Vanhoucke, V., et al.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, 4–9 February 2017, pp. 4278–4284 (2017)

  13. Szegedy, C., Liu, W., Jia, et al.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, pp. 1–9 (2015). https://doi.org/10.1109/CVPR.2015.7298594

  14. Yu, L., Yang, X., Hao, C., et al.: Volumetric ConvNets with mixed residual connections for automated prostate segmentation from 3D MR images. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, 4–9 February 2017, San Francisco, California, USA (2017)

  15. Shen, W., Wang, X., Wang, Y., Bai, X., Zhang, Z.: DeepContour: a deep convolutional feature learned by positive-sharing loss for contour detection. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, pp. 3982–3991 (2015)

  16. Isola, P., Zoran, D., Krishnan, D., Adelson, E.H.: Crisp boundary detection using pointwise mutual information. ECCV 8691, 799–814 (2014)

    Google Scholar 

  17. Bertasius, G., Shi, J., Torresani, L.: DeepEdge: a multi-scale bifurcated deep network for top-down contour detection. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) 2015, 4380–4389 (2015)

    Google Scholar 

  18. Dollár, P., Zitnick, C.L.: Fast edge detection using structured forests. IEEE Trans. Pattern Anal. Mach. Intell. 37(8), 1558–1570 (2015). https://doi.org/10.1109/TPAMI.2014.2377715

    Article  Google Scholar 

  19. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986). https://doi.org/10.1109/TPAMI.1986.4767851

    Article  Google Scholar 

  20. Fram, J.R., Deutsch, E.S.: On the quantitative evaluation of edge detection schemes and their comparison with human performance. IEEE Trans. Comput. 24(6), 616–628 (1975). https://doi.org/10.1109/T-C.1975.224274

    Article  MATH  Google Scholar 

  21. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990)

    Article  Google Scholar 

  22. Mottaghi, R., et al.: The role of context for object detection and semantic segmentation in the wild. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, pp. 891–898 (2014). https://doi.org/10.1109/CVPR.2014.119

  23. Gupta, S., Arbeláez, P., Malik, J.: Perceptual organization and recognition of indoor scenes from RGB-D images. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, pp. 564–571 (2013). https://doi.org/10.1109/CVPR.2013.79

  24. Le, T., Duan, Y.: REDN: a recursive encoder-decoder network for edge detection. IEEE Access 8, 90153–90164 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Beijing Jiaotong University, Beijing, 100044, China

    Yin Wang, Lide Wang, Ji Qiu & Yueyi Yang

Authors
  1. Yin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lide Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ji Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yueyi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yin Wang.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the Fundamental Research of the China State Railway Group Co., Ltd. under Grant N2020J007.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Wang, L., Qiu, J. et al. Feature enhancement: predict more detailed and crisper edges. SIViP 15, 1635–1642 (2021). https://doi.org/10.1007/s11760-021-01899-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-01899-1

Keywords

Search

Navigation