Skip to main content
Log in

Scene classification-oriented saliency detection via the modularized prescription

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Saliency detection technology has been greatly developed and applied in recent years. However, the performance of current methods is not satisfactory in complex scenes. One of the reasons is that the performance improvement is often carried out through utilizing complicated mathematical models and involving multiple features rather than classifying the scene complexity and respectively detecting saliency. To break this unified detection schema for generating better results, we propose a method of scene classification-oriented saliency detection via the modularized prescription in this paper. Different scenes are described by a scene complexity expression model, and they are analyzed and discriminately detected by different pipelines. This process seems like that doctors can tailor the treatment prescriptions when they meet different symptoms. Moreover, two SVM-based classifiers are trained for scene classification and sky region identification, and the proposed sky region discrimination and erase model can be used to efficiently decrease the saliency interference by the high luminance of the background sky regions. Experimental results demonstrate the effectiveness and superiority of the proposed method in both higher precision and better smoothness, especially for detecting in structure complex scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Achanta, R., Hemamiz, S., Estraday, F., et al.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1597–1604 (2009)

  2. Achanta, R., Shaji, A., Smith, K., et al.: Slic superpixels. Tech. Rep. (2010)

  3. Achanta, R., Susstrunk, S.: Saliency detection using maximum symmetric surround. In: 17th IEEE International Conference on Image Processing (ICIP), pp. 2653–2656 (2010)

  4. Ando, T.: Majorization relations for hadamard products. Linear Algebra Appl. 223–224(1), 57–64 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chen, Z.H., Liu, Y., Xiao, X.L., et al.: Moving visual focus in salient object segmentation. IET Image Proc. 9(9), 758–769 (2015)

    Article  Google Scholar 

  6. Cheng, M.M., Mitra, N.J., Huang, X., et al.: Salientshape: group saliency in image collections. Vis. Comput. 30(4), 443–453 (2014)

    Article  Google Scholar 

  7. Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)

    Article  Google Scholar 

  8. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Adv. Neural Inf. Process. Syst. 19, 545–552 (2006)

    Google Scholar 

  9. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007)

  10. Huang, Y., Fu, K., Yao, L., et al.: Saliency detection based on spread pattern and manifold ranking. In: 6th Chinese Conference on Pattern Recognition (CCPR), vol. 483, pp. 283–292 (2014)

  11. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  12. Judd, T., Durand, F., Torralba, A.: A benchmark of computational models of saliency to predict human fixations. In: MIT Technical Report (2012)

  13. Li, H., Chen, J., Lu, H., Chi, Z.: Cnn for saliency detection with low-level feature integration. Neurocomputing 226(C), 212–220 (2017)

    Article  Google Scholar 

  14. Li, X., Lu, H., Zhang, L., et al.: Saliency detection via dense and sparse reconstruction. In: IEEE International Conference on Computer Vision, pp. 2976–2983 (2013)

  15. Liu, J., Wang, S.: Salient region detection via simple local and global contrast representation. Neurocomputing 147, 435–443 (2015)

    Article  Google Scholar 

  16. Liu, T., Yuan, Z., Sun, J., et al.: Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 353–367 (2011)

    Article  Google Scholar 

  17. Margolin, R., Tal, A., Zelnik-Manor, L.: What makes a patch distinct? In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1139–1146 (2013)

  18. Margolin, R., Zelnik-Manor, L., Tal, A.: Saliency for image manipulation. Vis. Comput. 29(5), 381–392 (2013)

    Article  Google Scholar 

  19. Peng, H., Li, B., Ling, H., et al.: Salient object detection via structured matrix decomposition. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 818–832 (2017)

    Article  Google Scholar 

  20. Perazzi, F., Krahenbuhl, P., Pritch, Y., et al.: Saliency filters: Contrast based filtering for salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 733–740. (2012)

  21. Qiu, Y., Sun, X., She, M.F.: Saliency detection using hierarchical manifold learning. Neurocomputing 168, 538–549 (2015)

    Article  Google Scholar 

  22. Scharfenberger, C., Wong, A., Fergani, K., et al.: Statistical textural distinctiveness for salient region detection in natural images. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 979–986. (2013)

  23. Sharon, A., Meirav, G., Achi, B., et al.: Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 1–8 (2007)

    Google Scholar 

  24. Shen, X., Wu, Y.: A unified approach to salient object detection via low rank matrix recovery. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 853–860 (2012)

  25. Shi, J., Yan, Q., Xu, L., et al.: Hierarchical image saliency detection on extended cssd. IEEE Trans. Pattern Anal. Mach. Intell. 9(4), 1–14 (2014)

    Google Scholar 

  26. Song, M., Chen, C., Wang, S., et al.: Low-level and high-level prior learning for visual saliency estimation. Inf. Sci. 281, 573–585 (2014)

    Article  Google Scholar 

  27. Tilke, J., Ehinger, K., Durand, F., et al.: Learning to predict where humans look. In: IEEE International Conference on Computer Vision (ICCV), pp. 2106–2113 (2009)

  28. Tong, N., Lu, H., Xiang, R., et al.: Salient object detection via bootstrap learning. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1884–1892 (2015)

  29. Tong, N., Lu, H., Zhang, L., et al.: Saliency detection with multi-scale superpixels. IEEE Signal Process. Lett. 21(9), 1035–1039 (2014)

    Article  Google Scholar 

  30. Wang, L., Lu, H., Xiang, R., Yang, M.H.: Deep networks for saliency detection via local estimation and global search. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3183–3192 (2015)

  31. Wei, Y., Wen, F., Zhu, W., et al.: Geodesic saliency using background priors. In: European Conference on Computer Vision, pp. 29–42 (2012)

  32. Wu, X., Wang, H., Chen, W.: Saliency detection based on graph and independent component analysis with reference. In: 13th International Conference on Control Automation Robotics and Vision (ICARCV), pp. 1207–1212 (2014)

  33. Xu, M., Jiang, L., Ye, Z., et al.: Bottom-up saliency detection with sparse representation of learnt texture atoms. Pattern Recogn. 60, 348–360 (2016)

    Article  Google Scholar 

  34. Xu, M., Zhang, H.: Saliency detection with color contrast based on boundary information and neighbors. Vis. Comput. 31(3), 355–364 (2015)

    Article  MathSciNet  Google Scholar 

  35. Yang, C., Pu, J., Dong, Y., et al.: Salient object detection in complex scenes via d-s evidence theory based region classification. Vis. Comput. 33(11), 1415–1428 (2017)

    Article  Google Scholar 

  36. Yang, C., Pu, J., Xie, G.S., et al.: Extended locality-constrained linear self-coding for saliency detection. IEEE Signal Process. Lett. 24(10), 1458–1462 (2017)

    Article  Google Scholar 

  37. Yang, C., Zhang, L., Lu, H.: Graph-regularized saliency detection with convex-hull-based center prior. IEEE Signal Process. Lett. 20(7), 637–640 (2013)

    Article  Google Scholar 

  38. Yang, C., Zhang, L., Lu, H., et al.: Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3166–3173 (2013)

  39. Zhang, Y., Mao, Z., Li, J., et al.: Salient region detection for complex background images using integrated features. Inf. Sci. 281, 586–600 (2014)

    Article  Google Scholar 

  40. Zhu, S.S., Yung, N.H.C.: Sub-scene generation: a step towards complex scene understanding. In: IEEE International Conference on Multimedia, pp. 1–6 (2011)

  41. Zhu, W., Liang, S., Wei, Y., et al.: Saliency optimization from robust background detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2814–2821 (2014)

Download references

Acknowledgements

This work was supported in part by the International S & T Cooperation Program of Henan (No. 162102410021, 152102410036) and the National Natural Science Foundation of China (No. U1604153, U1504610).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jiexin Pu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, C., Pu, J., Dong, Y. et al. Scene classification-oriented saliency detection via the modularized prescription. Vis Comput 35, 473–488 (2019). https://doi.org/10.1007/s00371-018-1475-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1475-0

Keywords

Navigation