Skip to main content
SpringerLink
Log in
Book cover

International Conference on Artificial Neural Networks

ICANN 2020: Artificial Neural Networks and Machine Learning – ICANN 2020 pp 699–710Cite as

Enriched Feature Representation and Combination for Deep Saliency Detection

  • Conference paper
  • First Online:

  • 2964 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12396))

Abstract

One of the most challenging issue in visual saliency detection is to discover and integrate meaningful features through deep neural networks. Saliency detection model should be carefully designed to extract sufficient features from different levels and reorganize them into the final prediction. In this paper, we propose an efficient saliency detection framework by introducing multi-scale representation and multi-level combination to deep convolutional neural networks. The main idea of our proposed model is to optimize intra-level feature extraction and inter-level feature combination, so that both saliency semantic and object details can be correctly preserved in final saliency maps. The model utilizes parallel dilated convolutions and pyramid pooling structures to enhance local details and acquire multi-scale feature representation. Feature maps of different resolutions are integrated by performing hierarchical combination in the encoder and decoder parts respectively. As a result, the model can better retain detail information during feature extraction and locate salient regions for saliency map recovery. Experimental results show that our model achieves state-of-the-art performance on several representative datasets.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Gao, Y., Wang, M., Tao, D., Ji, R., Dai, Q.: 3-D Object Retrieval and Recognition With Hypergraph Analysis. IEEE Trans. Image Process. 21(9), 4290–4303 (2012)

    Article  MathSciNet  Google Scholar 

  2. Borji, A., Frintrop, S., Sihite, D.N., Itti, L.: Adaptive object tracking by learning background context. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 23–30, Providence, RI (2012)

    Google Scholar 

  3. Wang, Z., Chen, T., Li, G., Xu, R., Lin, L.: Multi-label image recognition by recurrently discovering attentional regions. In: 16th IEEE International Conference on Computer Vision, pp. 464–472. IEEE Press, Venice (2017)

    Google Scholar 

  4. Zhang, G.X., Cheng, M.M., Hu, S.M., Martin, R.R.: A Shape-preserving Approach to Image Resizing. Comput. Graph. Forum 28(7), 1897–1906 (2009)

    Article  Google Scholar 

  5. Wei, Y., Feng, J., Liang, X., Cheng, M.M., Zhao, Y., Yan, S.: Object region mining with adversarial erasing: a simple classification to semantic segmentation approach. In: 30th IEEE Conference on Computer Vision and Pattern Recognition, pp. 6488–6496. IEEE Press, Honolulu (2017)

    Google Scholar 

  6. Itti, L., Koch, C., Niebur, E.: A Model of Saliency-based Visual Attention for Rapid Scene Analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  7. Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: 22th IEEE Conference on Computer Vision and Pattern Recognition, pp. 1597–1604. IEEE Press, Miami (2009)

    Google Scholar 

  8. Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global Contrast Based Salient Region Detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 409–416 (2011)

    Google Scholar 

  9. Li, X., Lu, H., Zhang, L., Xiang, R., Yang, M.H.: Saliency detection via dense and sparse reconstruction. In: 26th IEEE Conference on Computer Vision and Pattern Recognition, pp. 2976–2983. IEEE Press, Portland (2013)

    Google Scholar 

  10. Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., Li, S.: Salient object detection: a discriminative regional feature integration approach. In: 26th IEEE Conference on Computer Vision and Pattern Recognition, pp. 2083–2090. IEEE Press, Portland (2013)

    Google Scholar 

  11. He, S., Lau, R.W.H., Liu, W., Huang, Z., Yang, Q.: SuperCNN: a superpixelwise convolutional neural network for salient object detection. Int. J. Comput. Vis. 115(3), 330–344 (2015). https://doi.org/10.1007/s11263-015-0822-0

    Article  MathSciNet  Google Scholar 

  12. Zhao, R., Ouyang, W., Li, H., Wang, X.: Saliency detection by multi-context deep learning. In: 28th IEEE Conference on Computer Vision and Pattern Recognition, pp. 1265–1274. IEEE Press, Boston (2015)

    Google Scholar 

  13. Lee, G., Tai, Y.W., Kim, J.: Deep saliency with encoded low level distance map and high level features. In: 29th IEEE Conference on Computer Vision and Pattern Recognition, pp. 660–668. IEEE Press, Las Vegas (2016)

    Google Scholar 

  14. Li, X., Zhao, L., Wei, L., Yang, M.H.: DeepSaliency: multi-task deep neural network model for salient object detection. IEEE Trans. Image Process. 25(8), 3919–3930 (2016)

    Article  MathSciNet  Google Scholar 

  15. Luo, Z., Mishra, A., Achkar, A., Eichel, J., Li, S.Z., Jodoin, P.M.: Non-local deep features for salient object detection. In: 30th IEEE Conference on Computer Vision and Pattern Recognition, pp. 6593–6601. IEEE Press, Honolulu (2017)

    Google Scholar 

  16. Zhang, P., Wang, D., Lu, H., Wang, H., Yin, B.: Learning uncertain convolutional features for accurate saliency detection. In: 16th IEEE International Conference on Computer Vision, pp. 212–221. IEEE Press, Venice (2017)

    Google Scholar 

  17. Zhang, P., Wang, D., Lu, H., Wang, H., Ruan, X.: Amulet: aggregating multi-level convolutional features for salient object detection. In: 16th IEEE International Conference on Computer Vision, pp. 202–211. IEEE Press, Venice (2017)

    Google Scholar 

  18. Li, G., Yu, Y.: Contrast-oriented deep neural networks for salient object detection. IEEE Trans. Neural Netw. Learn. Syst. 29(12), 6038–6051 (2018)

    Article  Google Scholar 

  19. Wang, L., Wang, L., Lu, H., Zhang, P., Ruan, X.: Salient object detection with recurrent fully convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1734–1746 (2019)

    Article  Google Scholar 

  20. Hou, Q., Cheng, M.M., Hu, X., Borji, A., Tu, Z., Torr, P.H.S.: Deeply supervised salient object detection with short connections. IEEE Trans. Pattern Anal. Mach. Intell. 41(4), 815–828 (2019)

    Article  Google Scholar 

  21. Wang, T., et al.: Detect globally, refine locally: a novel approach to saliency detection. In: 31th IEEE Conference on Computer Vision and Pattern Recognition, pp. 3127–3135. IEEE Press, Salt Lake City (2018)

    Google Scholar 

  22. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 28th IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440. IEEE Press, Boston (2015)

    Google Scholar 

  23. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 29th IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE Press, Las Vegas (2016)

    Google Scholar 

  24. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 30th IEEE Conference on Computer Vision and Pattern Recognition, pp. 6230–6239. IEEE Press, Honolulu (2017)

    Google Scholar 

  25. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  26. Krähenbühl, P., Koltun, V.: Efficient inference in fully connected CRFs with gaussian edge potentials. In: Advances in Neural Information Processing Systems (2011)

    Google Scholar 

  27. Wang, L., et al.: Learning to detect salient objects with image-level supervision. In: 29th IEEE Conference on Computer Vision and Pattern Recognition, pp. 3796–3805. IEEE Press, Honolulu (2017)

    Google Scholar 

  28. Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: 26th IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1162. IEEE Press, Portland (2013)

    Google Scholar 

  29. Li, Y., Hou, X., Koch, C., Rehg, J.M., Yuille, A.L.: The secrets of salient object segmentation. In: 27th IEEE Conference on Computer Vision and Pattern Recognition, pp. 280–287. IEEE Press, Columbus (2014)

    Google Scholar 

  30. Liu, T., Sun, J., Zheng, N., Tang, X., Shum, H.: Learning to detect a salient object. In: 20th IEEE Conference on Computer Vision and Pattern Recognition, pp. 353–367. IEEE Press, Minneapolis (2007)

    Google Scholar 

  31. Li, G., Yu, Y.: Visual saliency based on multiscale deep features. In: 27th IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463. IEEE Press, Boston (2015)

    Google Scholar 

  32. Alpert, S., Galun, M., Basri, R., Brandt, A.: Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 315–327 (2012)

    Article  Google Scholar 

  33. Cornia, M., Baraldi, L., Serra, G., Cucchiara, R.: A deep multi-level network for saliency prediction. In: 23th International Conference on Pattern Recognition, pp. 3488–3493. Amsterdam (2016)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by National Natural Science Foundation of China under grant 61771145 and 61371148.

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Fudan University, Shanghai, 200433, China

    Lecheng Zhou & Xiaodong Gu

Authors
  1. Lecheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaodong Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaodong Gu .

Editor information

Editors and Affiliations

  1. Department of Applied Informatics, Comenius University in Bratislava, Bratislava, Slovakia

    Igor Farkaš

  2. Department of Applied Mathematics and Computer Science, Technical University of Denmark, Kgs. Lyngby, Denmark

    Paolo Masulli

  3. Department of Informatics, University of Hamburg, Hamburg, Germany

    Stefan Wermter

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhou, L., Gu, X. (2020). Enriched Feature Representation and Combination for Deep Saliency Detection. In: Farkaš, I., Masulli, P., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2020. ICANN 2020. Lecture Notes in Computer Science(), vol 12396. Springer, Cham. https://doi.org/10.1007/978-3-030-61609-0_55

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-61609-0_55

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61608-3

  • Online ISBN: 978-3-030-61609-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation