Skip to main content
SpringerLink
Log in

Visual saliency detection via combining center prior and U-Net

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

At present, poor background suppression is one major problem for visual saliency detection. Although many mainstream saliency detection models can effectively locate salient objects, objects in complicated backgrounds in some natural images are often mistaken for salient objects. Therefore, this paper proposed to set up a center prior-based encoder-decoder network to improve background suppression results. A traditional center prior-based method and U-Net model were combined efficiently. First, multi-scale group convolution was used to replace general convolution, which can highlight the semantic information characteristics, and high-level characteristics at the bottom of U-Net were integrated and optimized on the basis of the consideration of center prior. Then, refinements were delivered throughout the whole network by upgrading the network structure, so as to ensure the optimized features can be made full use of. Since the changes to the U-Net architecture somewhat affected the stability of the network. Therefore, branch network modules were adopted and adaptive parameters defined to coordinate the relationships between branch networks to keep the network structure well balanced. The method has been tested with four widely used databases and proven effective by comparing its results with those of another seven popular methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Zünd, F., Pritch, Y., Sorkine-Hornung, A., et al.: Content-aware compression using saliency-driven image retargeting[C]. In: 2013 IEEE International Conference on Image Processing. IEEE, pp. 1845–1849 (2013)

  2. Qi, M., Wang, Y.: Deep-CSSR: Scene classification using category-specific salient region with deep features[C]. In: 2016 IEEE International Conference on Image Processing (ICIP). IEEE, pp. 1047–1051 (2016)

  3. Borji, A., Itti, L.: State-of-the-art in visual attention modeling[J]. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 185–207 (2012)

    Article  Google Scholar 

  4. Singh, N.: Saliency threshold: a novel saliency detection model using Ising’s theory on Ferromagnetism (STIF). Multimedia Syst. 26, 397–411 (2020)

    Article  Google Scholar 

  5. Guo, J., Ren, T., Huang, L., et al.: Saliency detection on sampled images for tag ranking. Multimedia Syst. 25, 35–47 (2019). https://doi.org/10.1007/s00530-017-0546-9

    Article  Google Scholar 

  6. Zhao, S., Shen, J., Li, F.: A hierarchical visual saliency detection method by combining distinction and background probability maps. Multimedia Syst. 23, 343–350 (2017). https://doi.org/10.1007/s00530-015-0490-5

    Article  Google Scholar 

  7. Sun, Y., Zhao, M., Hu, K., et al.: Visual saliency prediction using multi-scale attention gated network. Multimedia Syst. (2021). https://doi.org/10.1007/s00530-021-00796-4

    Article  Google Scholar 

  8. Itti, Laurent , C. Koch , and E. Niebur . "A model of saliency-based visual attention for rapid scene analysis." IEEE Transactions on Pattern Analysis & Machine Intelligence 20.11(2002):1254–1259.

  9. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition[J] (2014). arXiv preprint arXiv:1409.1556

  10. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  11. Ronneberger, O., Fischer, P., Brox, T.: Unet Convol-utional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, pp. 234–241 (2015)

  12. Zhang, P., Liu, W., Lu, H., et al.: Salient object detection with lossless feature reflection and weighted structural loss[J]. IEEE Trans. Image Process. 28(6), 3048–3060 (2019)

    Article  MathSciNet  Google Scholar 

  13. Wang, Q., Chen, J., Deng, J., et al.: 3D-CenterNet: 3D object detection network for point clouds with center estimation priority[J]. Pattern Recogn. 115, 107884 (2021)

    Article  Google Scholar 

  14. Zhang, J., Wang, M., Zhang, S., et al.: Spatiochromatic context modeling for color saliency analysis[J]. IEEE Trans. Neural Netw. Learn. Syst. 27(6), 1177–1189 (2015)

    Article  MathSciNet  Google Scholar 

  15. Liu, J., Wang, S.: Salient region detection via simple local and global contrast representation. Neurocomputing 147, 435–443 (2015). https://doi.org/10.1016/j.neucom.2014.06.041

    Article  Google Scholar 

  16. Zhao, R., Ouyang, W., Li, H., et al.: Saliency detection by multi-context deep learning[C]. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2015)

  17. Li, G., Xie, Y., Lin, L., et al.: Instance-level salient object segmentation[C]. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2017)

  18. Li, G., Yu, Y.: Visual saliency based on multiscale deep features[C]. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2015)

  19. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

  20. Wang, L., Wang, L., Lu, H., et al.: Saliency detection with recurrent fully convolutional networks[C]. ECCV 2016, PT IV, 825-841 (2016)

  21. Hu, P., Shuai, B., Liu, J., et al.: deep level sets for salient object detection[C]. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2017)

  22. Wu, Z., Su, L., Huang, Q.: Stacked cross refinement network for edge-aware salient object detection[C]. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE (2020)

  23. Qin, X., Zhang, Z., Huang, C., et al.: BASNet: boundary-aware salient object detection[C]. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2019)

  24. Liu, J.J., Hou, Q., Cheng, M.M., et al.: A simple pooling-based design for real-time salient object detection[C]. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2019)

  25. Chen, Z., Xu, Q., Cong, R., et al.: Global context-aware progressive aggregation network for salient object detection[C]. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34(07), pp. 10599–10606 (2020)

  26. Tan, Z., Gu, X.: Depth scale balance saliency detection with connective feature pyramid and edge guidance[J]. Appl. Intell. 51, 1–18 (2021)

    Article  Google Scholar 

  27. Yang, C., Zhang, L., Lu, H.: Graph-regularized saliency detection with convex-hull-based center prior[J]. IEEE Signal Process. Lett. 20(7), 637–640 (2013)

    Article  Google Scholar 

  28. Zhou, C., Gui, S., Zhang, G., et al.: Social group behavior analysis model integrating multitask learning and convolutional neural network[J]. Wirel. Commun. Mob. Comput. 2021, 1–14 (2021)

    Google Scholar 

  29. Achanta†, R., Hemami‡, S., Estrada†, F., et al.: Frequency-tuned salient region detection[J] (2009)

  30. Perazzi, F., Krahenbuhl, P., Pritch, Y., et al.: Saliency filters: contrast based filtering for salient region detection[C]. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE (2012)

  31. Mnih, V., Hinton, G.E.: Learning to detect roads in high-resolution aerial images[C]. In: Computer Vision—ECCV 2010—11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5–11, 2010, Proceedings, Part VI. Springer, Berlin, Heidelberg (2010)

  32. Wang, L., Lu, H., Wang, Y., et al.: Learning to detect salient objects with image-level supervision[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 136–145 (2017)

  33. Yang, C., Zhang, L., Lu, H., et al.: Saliency detection via graph-based manifold ranking[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173 (2013)

  34. Yan, Q., Xu, L., Shi, J., et al.: Hierarchical saliency detection[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1162 (2013)

  35. Li, G., Yu, Y.: Visual saliency based on multiscale deep features[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463 (2015)

  36. Zhang, P., Wang, D., Lu, H., et al.: Amulet: aggregating multi-level convolutional features for salient object detection[C]. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE (2017)

  37. Wang, T., Zhang, L., Wang, S., et al.: Detect globally, refine locally: a novel approach to saliency detection[C]. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2018)

  38. Luo, Z., Mishra, A., Achkar, A., et al.: Non-local deep features for salient object detection[C]. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2017)

  39. Deng, Z., Hu, X., Zhu, L., et al.: R^3 Net: recurrent residual refinement network for saliency detection[C]. In: International Joint Conference on Artificial Intelligence (IJCAI), 2018 (2018)

  40. Chen, S., Tan, X., Wang, B., et al.: Reverse attention for salient object detection[C]. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 234–250 (2018)

  41. Zhang, P., Wang, D., Lu, H., et al.: Learning uncertain convolutional features for accurate saliency detection[C]. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 212–221 (2017)

  42. Zhou, H., Xie, X., Lai, J.H., et al.: Interactive two-stream decoder for accurate and fast saliency detection[C]. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9141–9150 (2020)

  43. Bruce, N., Tsotsos, J.: Saliency based on information maximization[C]. Adv. Neural Inf. Process. Syst. 18(3), 155–162 (2005)

    Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (NSFC) (61976123, 61601427); Taishan Young Scholars Program of Shandong Province; and Key Development Program for Basic Research of Shandong Province (ZR2020ZD44).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China

    Xiangwei Lu & Muwei Jian

  2. School of Information Science and Technology, Linyi University, Linyi, China

    Muwei Jian & Xing Wang

  3. School of Creative Technologies, University of Portsmouth, Portsmouth, UK

    Hui Yu

  4. Department of Computer Science and Technology, Ocean University of China, Qingdao, China

    Junyu Dong

  5. Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, Hong Kong, China

    Kin-Man Lam

Authors
  1. Xiangwei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muwei Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hui Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junyu Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kin-Man Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muwei Jian.

Additional information

Communicated by A. Liu.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, X., Jian, M., Wang, X. et al. Visual saliency detection via combining center prior and U-Net. Multimedia Systems 28, 1689–1698 (2022). https://doi.org/10.1007/s00530-022-00940-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-00940-8

Keywords

Search

Navigation