Skip to main content
SpringerLink
Log in

Region-based depth feature descriptor for saliency detection on light field

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper addresses the light field saliency detection problem via a multiple cue integration framework. By reinterpreting the usage of dark channels in estimating the amount of defocus, a novel region-based depth feature descriptor (RDFD) defined over the focal stack is proposed. Compared to the methods which utilize the depth map as another image channel, the RDFD can produce more informative saliency cues and make less restrictive assumptions on accurate depth map or focused clear images containing dark pixels. The proposed RDFD facilitates saliency detection in the following two respects: (1) the region-based depth contrast map can be computed by measuring a pair-wise distance between super-pixels with the proposed RDFD, (2) a spatial distribution prior in the 3D space (3D-SDP) can be obtained from such depth measurements to provide high-level semantic guidances, including the gradient-like distribution in depth and the object-biased prior in image plane. Both of them contribute to generating a contrast-based depth saliency map and refining a background-based color saliency map. Finally, these saliency maps are merged into a single map using a multi-layer cellular antomata (MCA) optimizer. Experimental results demonstrate that our method outperforms state-of-the-art techniques on the challenging light field saliency detection benchmark LFSD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Achanta R, Estrada F, Wils P, Süsstrunk S (2008) Salient region detection and segmentation. In: ICVS. Springer, pp 66–75

  2. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: CVPR, pp 1597–1604

  3. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. TPAMI 34 (11):2274–2282

    Article  Google Scholar 

  4. Alexe B, Deselaers T, Ferrari V (2012) Measuring the objectness of image windows. TPAMI 34(11):2189–2202

    Article  Google Scholar 

  5. Borji A, Cheng MM, Jiang H, Li J (2015) Salient object detection: a benchmark. TIP 24(12):5706–5722

    MathSciNet  MATH  Google Scholar 

  6. Bruce N, Tsotsos J (2006) Saliency based on information maximization. In: NIPS, pp 155–162

  7. Cheng MM, Mitra NJ, Huang X, Torr PH, Hu SM (2015) Global contrast based salient region detection. TPAMI 37(3):569–582

    Article  Google Scholar 

  8. Ciptadi A, Hermans T, Rehg JM (2013) An in depth view of saliency. In: BMVC, pp 112.1–112.11

  9. Ding Y, Xiao J, Yu J (2011) Importance filtering for image retargeting. In: CVPR, pp 89–96

  10. Donoser M, Urschler M, Hirzer M, Bischof H (2009) Saliency driven total variation segmentation. In: ICCV, pp 817–824

  11. Duan L, Wu C, Miao J, Qing L, Fu Y (2011) Visual saliency detection by spatially weighted dissimilarity. In: CVPR, pp 473–480

  12. He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. TPAMI 33(12):2341–2353

    Article  Google Scholar 

  13. Itti L (2004) Automatic foveation for video compression using a neurobiological model of visual attention. TIP 13(10):1304–1318

    Google Scholar 

  14. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. TPAMI 20(11):1254–1259

    Article  Google Scholar 

  15. Jiang B, Zhang L, Lu H, Yang C, Yang MH (2013) Saliency detection via absorbing markov chain. In: ICCV, pp 1665–1672

  16. Jiang H, Wang J, Yuan Z, Liu T, Zheng N, Li S (2011) Automatic salient object segmentation based on context and shape prior. In: BMVC, pp 110.1–110.12

  17. Jiang P, Ling H, Yu J, Peng J (2013) Salient region detection by ufo: Uniqueness, focusness and objectness. In: ICCV, pp 1976–1983

  18. Lang C, Nguyen TV, Katti H, Yadati K, Kankanhalli M, Yan S (2012) Depth matters: influence of depth cues on visual saliency. In: ECCV, pp 101–115

  19. Li N, Sun B, Yu J (2015) A weighted sparse coding framework for saliency detection. In: CVPR, pp 5216–5223

  20. Li N, Ye J, Ji Y, Ling H, Yu J (2014) Saliency detection on light field. In: CVPR, pp 2806–2813

  21. Li X, Lu H, Zhang L, Ruan X, Yang MH (2013) Saliency detection via dense and sparse reconstruction. In: ICCV, pp 2976–2983

  22. Liu A, Nie W, Gao Y, Su Y (2016) Multi-modal clique-graph matching for view-based 3d model retrieval. TIP 25(5):2103–2116

    MathSciNet  MATH  Google Scholar 

  23. Liu A, Nie W, Gao Y, Su Y (2018) View-based 3d model retrieval: a benchmark. IEEE Trans Cybern 48(3):916–928

    Google Scholar 

  24. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum HY (2011) Learning to detect a salient object. TPAMI 33(2):353–367

    Article  Google Scholar 

  25. Mai L, Niu Y, Liu F (2013) Saliency aggregation: a data-driven approach. In: CVPR, pp 1131–1138

  26. Okutomi M, Kanade T (1993) A multiple-baseline stereo. IEEE Trans Pattern Anal Mach Intell 15(4):353–363

    Article  Google Scholar 

  27. Pan J, Sun D, Pfister H, Yang MH (2016) Blind image deblurring using dark channel prior. In: CVPR, pp 1628–1636

  28. Peng H, Li B, Xiong W, Hu W, Ji R (2014) Rgbd salient object detection: a benchmark and algorithms. In: ECCV, pp 92–109

  29. Pentland AP (1987) A new sense for depth of fiel. TPAMI (4), 523–531

  30. Qin Y, Lu H, Xu Y, Wang H (2015) Saliency detection via cellular automata. In: CVPR, pp 110–119

  31. Ren J, Gong X, Yu L, Zhou W, Yang MY (2015) Exploiting global priors for rgb-d saliency detection. In: CVPR Workshops, pp 25–32

  32. Rutishauser U, Walther D, Koch C, Perona P (2004) Is bottom-up attention useful for object recognition?. In: CVPR, vol 2, pp II–II

  33. Shen X, Wu Y (2012) A unified approach to salient object detection via low rank matrix recovery. In: CVPR, pp 853–860

  34. Sheng H, Feng W, Zhang S (2016) Cellular automata based on occlusion relationship for saliency detection. In: KSEM, pp 28–39

  35. Sheng H, Zhang S, Liu X, Xiong Z (2016) Relative location for light field saliency detection. In: ICASSP, pp 1631–1635

  36. Wang W, Wang Y, Huang Q, Gao W (2010) Measuring visual saliency by site entropy rate. In: CVPR, pp 2368–2375

  37. Xue T, Owens A, Scharstein D, Goesele M, Szeliski R (2019) Multi-frame stereo matching with edges, planes and superpixels. Image Vis Comput 103771:91

    Google Scholar 

  38. Yan Q, Xu L, Shi J (2013) Hierarchical saliency detection. In: CVPR, pp 1155–1162

  39. Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: CVPR, pp 3166–3173

  40. Yang X, Qian X, Xue Y (2015) Scalable mobile image retrieval by exploring contextual saliency. TIP 24(6):1709–1721

    MathSciNet  MATH  Google Scholar 

  41. Zhang J, Wang M, Gao J, Wang Y, Zhang X, Wu X (2015) Saliency detection with a deeper investigation of light field. In: IJCAI, pp 2212–2218

  42. Zhang J, Wang M, Lin L, Yang X, Gao J, Rui Y (2017) Saliency detection on light field: a multi-cue approach. TOMM 13(3):32:1–32:22

    Article  Google Scholar 

  43. Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: CVPR, pp 2814–2821

  44. Zitnick C, Kang S, Uyttendaele M, Winder S, Szeliski R (2004) High-quality video view interpolation using a layered representation. In: SIGGRAPH, pp 600–608

Download references

Funding

This work is supported by NSFC under Grant 61801396 and 61531014, and by the Fundamental Research Funds for the Central Universities under Grant 3102018zy030.

Author information

Authors and Affiliations

  1. School of Computer Science, Northwestern Polytechnical University, Xi’an, 710072, People’s Republic of China

    Xue Wang, Yingying Dong, Qi Zhang & Qing Wang

Authors
  1. Xue Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingying Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xue Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Dong, Y., Zhang, Q. et al. Region-based depth feature descriptor for saliency detection on light field. Multimed Tools Appl 80, 16329–16346 (2021). https://doi.org/10.1007/s11042-020-08890-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08890-x

Keywords

Search

Navigation