Skip to main content
SpringerLink
Log in

Deep Learning Features Inspired Saliency Detection of 3D Images

  • Conference paper
  • First Online:
Advances in Multimedia Information Processing - PCM 2016 (PCM 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9917))

Included in the following conference series:

Abstract

Saliency detection of 3D images is important for many 3D applications, such as bit allocation in 3D video coding, spatial pooling in stereoscopic image quality assessment and feature extraction in 3D object retrieval. However, traditional saliency detection approaches only target for the 2D images. Meanwhile, the traditional hand-crafted low-level feature extraction process may be not suitable for the 3D images. In this paper, we propose a deep learning feature based 3D visual saliency detection model. The pre-trained CNN model is employed to extract the feature vectors for both color and depth images after multi-level image segmentation. Then, we train a neutral network based classifier to generate the color and depth saliency maps from the feature vectors. Final, the linear fusion method is adopted to obtain the final saliency map for 3D image. Experimental results demonstrate that our proposed model can achieve appealing performance improvement over two public benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Guo, C., Zhang, L.: A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans. Image Process. 19(1), 185–198 (2010)

    Article  MathSciNet  Google Scholar 

  2. Ma, L., Lin, W., Deng, C., Ngan, K.N.: Image retargeting quality assessment: a study of subjective scores and objective metrics. IEEE J. Sel. Top. Sign. Process. 6(6), 626–639 (2012)

    Article  Google Scholar 

  3. Ma, L., Li, S., Zhang, F., Ngan, K.N.: Reduced-reference image quality assessment using reorganized DCT-based image representation. IEEE Trans. Multimedia 13(4), 824–829 (2011)

    Article  Google Scholar 

  4. Fang, Y.M., Lin, W.S., Chen, Z.Z., Tsai, C.M., Lin, C.W.: A video saliency detection model in compressed domain. IEEE Trans. Circuits Syst. Video Technol. 24(1), 27–38 (2014)

    Article  Google Scholar 

  5. Fang, Y.M., Chen, Z.Z., Lin, W.S., Lin, C.W.: Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans. Image Process. 21(9), 3888–3901 (2012)

    Article  MathSciNet  Google Scholar 

  6. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  7. Song, X., Zhang, J., Han, Y., Jiang, J.: Semi-supervised feature selection via hierarchical regression for web image classification. Multimedia Syst. 22(1), 41–49 (2016)

    Article  Google Scholar 

  8. Zhang, J., Han, Y., Jiang, J.: Tensor rank selection for multimedia analysis. J. Vis. Commun. Image Represent. 30, 376–392 (2015)

    Article  Google Scholar 

  9. Fang, Y., Wang, J., Narwaria, M., Callet, P.L., Lin, W.: Saliency detection for stereoscopic images. IEEE Trans. Image Process. 23(6), 2625–2636 (2014)

    Article  MathSciNet  Google Scholar 

  10. Qi, F., Zhao, D., Liu, S., Fan, X.: 3D visual saliency detection model with generated disparity map. Multimedia Tools Appl. (2016). doi:10.1007/s11042-015-3229-6

    Google Scholar 

  11. Li, G., Yu, Y.: Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463 (2015)

    Google Scholar 

  12. Bruce, N., Tsotsos, J.: Saliency based on information maximization. In: Advances in Neural Information Processing Systems, pp. 155–162 (2005)

    Google Scholar 

  13. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

    Google Scholar 

  14. Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)

    Article  Google Scholar 

  15. Yang, J., Yang, M.H.: Top-down visual saliency via joint CRF and dictionary learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2296–2303 (2012)

    Google Scholar 

  16. Kanan, C., Tong, M.H., Zhang, L., Cottrell, G.W.: Sun: Top-down saliency using natural statistics. Visual Cogn. 17(6–7), 979–1003 (2009)

    Article  Google Scholar 

  17. Itti, L., Koch, C.: A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 40(10), 1489–1506 (2000)

    Article  Google Scholar 

  18. Cheng, M., Mitra, N.J., Huang, X., Torr, P.H., Hu, S.: Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2015)

    Article  Google Scholar 

  19. Zhao, R., Ouyang, W., Li, H., Wang, X.: Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1265–1274 (2015)

    Google Scholar 

  20. Wang, J., DaSilva, M.P., Callet, P.L., Ricordel, V.: Computational model of stereoscopic 3D visual saliency. IEEE Trans. Image Process. 22(6), 2151–2165 (2013)

    Article  MathSciNet  Google Scholar 

  21. Kim, H., Lee, S., Bovik, A.C.: Saliency prediction on stereoscopic videos. IEEE Trans. Image Process. 23(4), 1476–1490 (2014)

    Article  MathSciNet  Google Scholar 

  22. Song, H.A., Lee, S.-Y.: Hierarchical representation using NMF. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8226, pp. 466–473. Springer, Heidelberg (2013). doi:10.1007/978-3-642-42054-2_58

    Chapter  Google Scholar 

  23. Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., Li, S.: Salient object detection: a discriminative regional feature integration approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2083–2090 (2013)

    Google Scholar 

  24. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

    Google Scholar 

  25. İmamoğlu, N., Lin, W., Fang, Y.: A saliency detection model using low-level features based on wavelet transform. IEEE Trans. Multimedia 15(1), 96–105 (2013)

    Article  Google Scholar 

  26. Lang, C., Nguyen, T.V., Katti, H., Yadati, K., Kankanhalli, M., Yan, S.: Depth matters: influence of depth cues on visual saliency. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 101–115. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33709-3_8

    Chapter  Google Scholar 

  27. Ma, C.Y., Hang, H.M.: Learning-based saliency model with depth information. J. Vision 15(6), 19 (2015)

    Article  Google Scholar 

  28. Judd, T., Durand, F., Torralba, A.: A benchmark of computational models of saliency to predict human fixations. Massachusetts Inst. Technol., MA, USA, Computer Science and Artificial Intelligence Lab (CSAIL), Technical rep. MIT-CSAIL-TR-2012–001 (2012)

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 61501299 and 61373103, in part by the Guangdong Nature Science Foundation under Grant 2016A030310058, in part by the Shenzhen Emerging Industries of the Strategic Basic Research Project under Grants JCYJ20150525092941043 and JCYJ20160226191842793, in part by the Project 2016049 supported by SZU R/D Fund.

Author information

Authors and Affiliations

  1. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China

    Qiudan Zhang, Xu Wang & Jianmin Jiang

  2. Research Institute for Future Media Computing, Shenzhen University, Shenzhen, 518060, China

    Qiudan Zhang, Xu Wang & Jianmin Jiang

  3. Huawei Noah’s Ark Lab, Shatin, Hong Kong

    Lin Ma

Authors
  1. Qiudan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianmin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lin Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xu Wang .

Editor information

Editors and Affiliations

  1. Zhengzhou University, Zhengzhou, China

    Enqing Chen

  2. Jiaotong University, Xi’an, China

    Yihong Gong

  3. Zhengzhou University, Zhengzhou, China

    Yun Tie

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Zhang, Q., Wang, X., Jiang, J., Ma, L. (2016). Deep Learning Features Inspired Saliency Detection of 3D Images. In: Chen, E., Gong, Y., Tie, Y. (eds) Advances in Multimedia Information Processing - PCM 2016. PCM 2016. Lecture Notes in Computer Science(), vol 9917. Springer, Cham. https://doi.org/10.1007/978-3-319-48896-7_57

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-48896-7_57

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-48895-0

  • Online ISBN: 978-3-319-48896-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation