Skip to main content
SpringerLink
Log in

Efficient Depth-Included Residual Refinement Network for RGB-D Saliency Detection

  • Conference paper
  • First Online:
Image and Graphics (ICIG 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12890))

Included in the following conference series:

  • 2219 Accesses

Abstract

RGB-D saliency detection aims to segment eye-catching objects from images with the help of depth. Although many excellent methods raised, it is still difficult to locate salient objects accurately and efficiently, which lies in two challenges: (1) It is difficult to seamlessly and efficiently integrate cross-modal features from RGB-D inputs; (2) Low-quality depth maps have a serious negative impact on the final prediction results. The existing methods use two backbone networks to extract saliency features, which also introduce much redundancy. To address issues, we propose a simple and efficient deep feature refinement module to extract complementary depth features. We also design a depth correction module to filter out noisy depth input adaptively. Experiments with 13 recently proposed methods on 7 datasets demonstrate the effectiveness of the proposed approach both quantitatively and qualitatively, especially in efficiency and compactness.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Boer, P.T.D., Kroese, D., Mannor, S., Rubinstein, R.: A tutorial on the cross-entropy method. Ann. Oper. Res. 134, 19–67 (2002)

    Google Scholar 

  2. Borji, A., Cheng, M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)

    Article  MathSciNet  Google Scholar 

  3. Chen, H., Li, Y.: Three-stream attention-aware network for rgb-d salient object detection. IEEE Trans. Image Process. 28(6), 2825–2835 (2019)

    Article  MathSciNet  Google Scholar 

  4. Chen, H., Li, Y.: Progressively complementarity-aware fusion network for rgb-d salient object detection, pp. 3051–3060 (06 2018)

    Google Scholar 

  5. Chen, H., Li, Y., Su, D.: Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for rgb-d salient object detection. Pattern Recogn. 86, 376–385 (2018)

    Google Scholar 

  6. Chen, S., Fu, Y.: Progressively guided alternate refinement network for RGB-D salient object detection. CoRR abs/2008.07064 (2020)

    Google Scholar 

  7. Chen, Z., Cong, R., Xu, Q., Huang, Q.: Dpanet: Depth potentiality-aware gated attention network for rgb-d salient object detection. IEEE Trans. Image Process. 1, 7012–7024 (2020)

    Google Scholar 

  8. Fan, D.P., Lin, Z., Zhang, Z., Zhu, M., Cheng, M.M.: Rethinking rgb-d salient object detection: models, data sets, and large-scale benchmarks. IEEE Trans. Neural Netw. Learn. Syst. 1, 2075–2089 (2020)

    Google Scholar 

  9. Fan, D.P., Gong, C., Cao, Y., Ren, B., Cheng, M.M., Borji, A.: Enhanced-alignment measure for binary foreground map evaluation (2018)

    Google Scholar 

  10. Fu, K., Fan, D.P., Ji, G.P., Zhao, Q.: Jl-dcf: Joint learning and densely-cooperative fusion framework for rgb-d salient object detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3049–3059 (2020)

    Google Scholar 

  11. Hong, S., You, T., Kwak, S., Han, B.: Online tracking by learning discriminative saliency map with convolutional neural network (2015)

    Google Scholar 

  12. Ju, R., Ge, L., Geng, W., Ren, T., Wu, G.: Depth saliency based on anisotropic center-surround difference. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1115–1119 (2014)

    Google Scholar 

  13. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)

    Google Scholar 

  14. Li, G., Liu, Z., Ling, H.: Icnet: Information conversion network for rgb-d based salient object detection. IEEE Trans. Image Process. 29, 4873–4884 (2020)

    Article  Google Scholar 

  15. Li, G., Zhu, C.: A three-pathway psychobiological framework of salient object detection using stereoscopic technology. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 3008–3014 (2017)

    Google Scholar 

  16. Li, N., Ye, J., Ji, Y., Ling, H., Yu, J.: Saliency detection on light field. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, pp. 2806–2813. IEEE Computer Society, USA (2014)

    Google Scholar 

  17. Liu, J., Hou, Q., Cheng, M., Feng, J., Jiang, J.: A simple pooling-based design for real-time salient object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3912–3921 (2019)

    Google Scholar 

  18. Liu, N., Zhang, N., Han, J.: Learning selective self-mutual attention for rgb-d saliency detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13753–13762 (2020)

    Google Scholar 

  19. Máttyus, G., Luo, W., Urtasun, R.: Deeproadmapper: extracting road topology from aerial images. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3458–3466 (2017)

    Google Scholar 

  20. Niu, Y., Geng, Y., Li, X., Liu, F.: Leveraging stereopsis for saliency analysis. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 454–461 (2012)

    Google Scholar 

  21. Pang, Y., Zhang, L., Zhao, X., Lu, H.: Hierarchical dynamic filtering network for rgb-d salient object detection (2020)

    Google Scholar 

  22. Piao, Y., Ji, W., Li, J., Zhang, M., Lu, H.: Depth-induced multi-scale recurrent attention network for saliency detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7253–7262 (2019)

    Google Scholar 

  23. Piao, Y., Rong, Z., Zhang, M., Ren, W., Lu, H.: A2dele: adaptive and attentive depth distiller for efficient rgb-d salient object detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9057–9066 (2020)

    Google Scholar 

  24. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (09 2014)

    Google Scholar 

  25. Su, J., Li, J., Zhang, Y., Xia, C., Tian, Y.: Selectivity or invariance: boundary-aware salient object detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3798–3807 (2019)

    Google Scholar 

  26. Wang, W., Shen, J., Yang, R., Porikli, F.: Saliency-aware video object segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 20–33 (2018)

    Article  Google Scholar 

  27. Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: 2003 The Thrity-Seventh Asilomar Conference on Signals, Systems Computers, vol. 2, pp. 1398–1402 (2003)

    Google Scholar 

  28. Zeng, Y., Zhuge, Y., Lu, H., Zhang, L., Qian, M., Yu, Y.: Multi-source weak supervision for saliency detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6067–6076 (2019)

    Google Scholar 

  29. Zhang, F., Du, B., Zhang, L.: Saliency-guided unsupervised feature learning for scene classification. IEEE Trans. Geosci. Remote Sens. 53(4), 2175–2184 (2015)

    Article  Google Scholar 

  30. Zhang, J., et al.: Uc-net: Uncertainty inspired rgb-d saliency detection via conditional variational autoencoders. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8579–8588 (2020)

    Google Scholar 

  31. Zhang, M., Ren, W., Piao, Y., Rong, Z., Lu, H.: Select, supplement and focus for rgb-d saliency detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3469–3478 (2020)

    Google Scholar 

  32. Zhao, J., Cao, Y., Fan, D., Cheng, M., Li, X., Zhang, L.: Contrast prior and fluid pyramid integration for rgbd salient object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3922–3931 (2019)

    Google Scholar 

  33. Zhao, R., Ouyang, W., Wang, X.: Unsupervised salience learning for person re-identification. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3586–3593 (2013)

    Google Scholar 

  34. Zhao, T., Wu, X.: Pyramid feature attention network for saliency detection (2019)

    Google Scholar 

  35. Zhao, X., Pang, Y., Zhang, L., Lu, H., Zhang, L.: Suppress and balance: a simple gated network for salient object detection (2020)

    Google Scholar 

  36. Zhao, X., Zhang, L., Pang, Y., Lu, H., Zhang, L.: A single stream network for robust and real-time rgb-d salient object detection (2020)

    Google Scholar 

  37. Zhu, C., Cai, X., Huang, K., Li, T.H., Li, G.: Pdnet: prior-model guided depth-enhanced network for salient object detection. In: 2019 IEEE International Conference on Multimedia and Expo (ICME), pp. 199–204 (2019)

    Google Scholar 

Download references

Acknowledgement

Supported by the Natural Science Foundation of China (No. 61802336 No. 61806175 No. 62073322), Jiangsu Province 7th Projects for Summit Talents in Six Main Industries, Electronic Information Industry (DZXX-149, No.110), Yangzhou University “Qinglan Project”.

Author information

Authors and Affiliations

  1. School of Information Engineering, Yangzhou University, Yangzhou, China

    Jinhao Yu, Guoliang Yan, Xiuqi Xu, Jian Wang, Shuhan Chen & Xuelong Hu

Authors
  1. Jinhao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guoliang Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiuqi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuhan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xuelong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuhan Chen .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Yuxin Peng

  2. Tsinghua University, Beijing, China

    Shi-Min Hu

  3. Tampere University, Tampere, Finland

    Moncef Gabbouj

  4. Zhejiang University, Hangzhou, China

    Kun Zhou

  5. Technion – Israel Institute of Technology, Haifa, Israel

    Michael Elad

  6. Tsinghua University, Beijing, China

    Kun Xu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yu, J., Yan, G., Xu, X., Wang, J., Chen, S., Hu, X. (2021). Efficient Depth-Included Residual Refinement Network for RGB-D Saliency Detection. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12890. Springer, Cham. https://doi.org/10.1007/978-3-030-87361-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87361-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87360-8

  • Online ISBN: 978-3-030-87361-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation