Skip to main content
SpringerLink
Log in

Prominent edge detection with deep metric expression and multi-scale features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Edge detection is one of today’s hottest computer vision issues with widely applications. It is beneficial for improving the capability of many vision systems, such as semantic segmentation, salient object detection and object recognition. Deep convolution neural networks (CNNs) recently have been employed to extract robust features, and have achieved a definite improvement. However, there is still a long run to study this hotspot with the main reason that CNNs-based approaches may cause the edges thicker. To address this problem, a novel semantic edge detection algorithm using multi-scale features is proposed. Our model is deep symmetrical metric learning network, which includes 3 key parts. Firstly, the deep detail layer, as a preprocessing layer and a guide module, is employed to remove some low-frequency information and still maintain the edge. Secondly, the deep encoder-decoder networks extract multi-scale features of original image, integrated for complementing information among each level feature. Finally, metric learning is introduced to generate a metric space used to predict edge result. It is easy to distinguish different categories, such as edge space and object space. Simulations and comparisons on benchmark datasets demonstrate the proposed algorithm is superior to the others through visual and quantitative evaluation, and specifically, the score of ODS reachs 0.788.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  2. Arbeláez P, Pont-Tuset J, Barron JT, Marques F, Malik J (2014) Multiscale combinatorial grouping. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 328–335

  3. Bertasius G, Shi J, Deepedge LT (2015) A multi-scale bifurcated deep network for top-down contour detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4380–4389

  4. Cai S, Huang J, Ding X, Zeng D (2017) Semantic edge detection based on deep metric learning. In: International symposium on intelligent signal proceeding and communication systems, pp 707–712

  5. Cheng M-M, Liu Y, Hou Q, Bian J, Torr P, Hu S-M, Tu Z (2016) Hfs: hierarchical feature selection for efficient image segmentation. In: European conference on computer vision. Springer, pp 867–882

  6. Deng J, Dong W, Socher R, Li L-J, Li K, Li F-F (2009) Imagenet: a large-scale hierarchical image database. In: CVPR 2009. IEEE conference on computer vision and pattern recognition, 2009. IEEE, pp 248–255

  7. Dollar P, Tu Z, Belongie S (2006) Supervised learning of edges and object boundaries. In: 2006 IEEE computer society conference on computer vision and pattern recognition, vol 2. IEEE, pp 1964–1971

  8. Dollár P, Zitnick CL (2013) Structured forests for fast edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1841–1848

  9. Dollár P, Zitnick CL (2015) Fast edge detection using structured forests. IEEE Trans Pattern Anal Mach Intell 37(8):1558–1570

    Article  Google Scholar 

  10. Ferrari V, Fevrier L, Jurie F, Schmid C (2008) Groups of adjacent contour segments for object detection. IEEE Trans Pattern Anal Mach Intell 30(1):36–51

    Article  Google Scholar 

  11. Ganin Y, Lempitsky V (2014) N 4-fields: neural network nearest neighbor fields for image transforms. In: Asian conference on computer vision. Springer, pp 536–551

  12. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

  13. Goldberger J, Hinton GE, Roweis ST, Salakhutdinov RR (2005) Neighbourhood components analysis. In: Advances in neural information processing systems, pp 513–520

  14. Hallman S, Fowlkes CC (2015) Oriented edge forests for boundary detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1732–1740

  15. Hastie T, Tibshirani R (1996) Discriminant adaptive nearest neighbor classification. IEEE Trans Pattern Anal Mach Intell 18(6):607–616

    Article  Google Scholar 

  16. He K, Sun J, Tang X (2013) Guided image filtering. IEEE Trans Pattern Anal Mach Intell 35(6):1397–1409

    Article  Google Scholar 

  17. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  18. Hu J, Lu J, Tan Y-P (2014) Discriminative deep metric learning for face verification in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1875–1882

  19. Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Li F-F (2014) Large-scale video classification with convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1725–1732

  20. Koestinger M, Hirzer M, Wohlhart P, Roth PM, Bischof H (2012) Large scale metric learning from equivalence constraints. In: 2012 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE, pp 2288–2295

  21. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  22. Li Z, Tang J (2015) Weakly supervised deep metric learning for community-contributed image retrieval. IEEE Trans Multimedia 17(11):1989–1999

    Article  Google Scholar 

  23. Lim JJ, Zitnick LC, Dollár P (2013) Sketch tokens: a learned mid-level representation for contour and object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3158–3165

  24. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37

  25. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  26. Maninis K-K, Pont-Tuset J, Arbeláez P, Gool LV (2016) Convolutional oriented boundaries. In: European conference on computer vision. Springer, pp 580–596

  27. Martin DR, Fowlkes CC, Malik J (2004) Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans Pattern Anal Mach Intell 26(5):530–549

    Article  Google Scholar 

  28. Nguyen HV, Bai L (2010) Cosine similarity metric learning for face verification. In: Asian conference on computer vision. Springer, pp 709–720

  29. Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 1520–1528

  30. Prewitt JMS (1970) Object enhancement and extraction. Picture Processing and Psychopictorics 10(1):15–19

    Google Scholar 

  31. Rastegari M, Ordonez V, Redmon J, Farhadi A (2016) Xnor-net: imagenet classification using binary convolutional neural networks. In: European conference on computer vision. Springer, pp 525–542

  32. Ren S, He K, Girshick R, Jian S (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

  33. Roberts LG (1963) Machine perception of three-dimensional solids. PhD thesis Massachusetts Institute of Technology

  34. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  35. Shen W, Wang X, Wang Y, Bai X, Zhang Z (2015) Deepcontour: a deep convolutional feature learned by positive-sharing loss for contour detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3982–3991

  36. Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from rgbd images. In: European conference on computer vision, pp 746–760

  37. Sobel I (1970) Camera models and machine perception. Technical report, Stanford Univ Calif Dept of Computer Science

  38. Uijlings JRR, De Sande KEAV, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171

    Article  Google Scholar 

  39. Wang L, Lu H, Ruan X, Yang M-H (2015) Deep networks for saliency detection via local estimation and global search. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 3183–3192

  40. Wei Y, Liang X, Chen Y, Shen X, Cheng M-M, Feng J, Zhao Y, Yan S (2017) Stc: a simple to complex framework for weakly-supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(11):2314–2320

    Article  Google Scholar 

  41. Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403

  42. Yang B, Zhang X, Li C, Yang H, Gao Z (2017) Edge guided salient object detection. Neurocomputing 221:60–71

    Article  Google Scholar 

  43. Yang Z, Xiang Y, Xie K, Lai Y (2017) Adaptive method for nonsmooth nonnegative matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 28(4):948–960

    Article  Google Scholar 

  44. Zhang W, Cham W-K (2012) Gradient-directed multiexposure composition. IEEE Trans Image Process 21(4):2318–2323

    Article  MathSciNet  MATH  Google Scholar 

  45. Zhang W, Ma B, Liu K, Huang R (2017) Video-based pedestrian re-identification by adaptive spatio-temporal appearance model. IEEE Trans Image Process 26(4):2042–2054

    Article  MathSciNet  MATH  Google Scholar 

  46. Zhang W, Yu X, He X (2017) Learning bidirectional temporal cues for video-based person re-identification. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2017.2718188

  47. Zhang Z, Kwok JT, Yeung D-Y (2003) Parametric distance metric learning with label information. In: IJCAI, p 1450

  48. Zhao R, Ouyang W, Li H, Wang X (2015) Saliency detection by multi-context deep learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1265–1274

  49. Zitnick CL, Dollár P (2014) Edge boxes: locating object proposals from edges. In: European conference on computer vision. Springer, pp 391–405

  50. Zou W, Komodakis N (2015) HARF: hierarchy-associated rich features for salient object detection. In: Proceedings of the IEEE international conference on computer vision. IEEE, pp 406–414

  51. Zou W, Liu Z, Kpalma K, Ronsin J, Zhao Yong, Komodakis N (2015) Unsupervised joint salient region detection and object segmentation. IEEE Trans Image Process 24(11):3858–3873

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work was supported in part from the grants of National Science Foundation of China (6151005, 61571382, 61103121, 81671766) and the funding from China Scholarship Council CSC NO. 201806155037, and open funding from Xiamen Key Laboratory of Mobile Multimedia Communications (Huaqiao University), and Guangdong Natural Science Foundation (2015A030313007, 2015A030313589).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Xiamen University, Xiamen, 361005, China

    Shulian Cai, Jiabin Huang, Yue Huang & Xinghao Ding

  2. Xiamen Key Laboratory of Mobile Multimedia Communications (Huaqiao University), Xiamen, 361021, China

    Shulian Cai & Jing Chen

  3. School of Mathematics, South China University of Technology, Guangzhou, 510641, China

    Delu Zeng

Authors
  1. Shulian Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiabin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yue Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xinghao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Delu Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Delu Zeng.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A preliminary version of this work appeared at ISPACS [4].

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, S., Huang, J., Chen, J. et al. Prominent edge detection with deep metric expression and multi-scale features. Multimed Tools Appl 78, 29121–29135 (2019). https://doi.org/10.1007/s11042-018-6581-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6581-5

Keywords

Search

Navigation