Skip to main content
Springer Nature Link
Log in

Saliency Detection via Sparse Reconstruction Errors of Covariance Descriptors on Riemannian Manifolds

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

We present a novel visual saliency detection method using covariance matrices on a Riemannian manifold. After over-segmentation, superpixels are generated and featured by the region covariance matrix. The superpixels on image boundary are regarded as possible background cues and are used to build the background dictionary. A sparse model is then constructed based on the background dictionary, where a kernel method, embedding Riemannian manifolds into reproducing kernel Hilbert space, is used. For each superpixel, we compute sparse reconstruction errors as a saliency measurement, which are then weighted based on the local context and global context information. Finally, multi-scale reconstruction errors are integrated to reduce the effect of the scale problem, and an object-biased Gaussian model is adopted to refine the saliency map. The main contribution of this paper is using a kernel sparse representation of the region covariance descriptors for saliency detection. Experiments with public benchmark dataset show that the proposed algorithm outperforms the state-of-the-art methods in terms of precision, recall, and mean absolute error, which demonstrate that our method is more effective in uniformly highlighting salient objects and is robust to background noise.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Susstrunk, Slic superpixels. EPFL, Lausssanne, Switzerland, Technical Report. 149300 (2010)

  2. V. Arsigny, P. Fillard, X. Pennec, N. Ayache, Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM J. Matrix Anal. Appl. 29(1), 328–347 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  3. R. Achanta, S. Hemami, F.J. Estrada, S. Süsstrunk, Frequency-tuned salient region detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2009), pp. 1597–1604

  4. S. Avidan, A. Shamir, Seam carving for content-aware image resizing. ACM Trans. Graph. 26(3), 10 (2007)

    Article  Google Scholar 

  5. R. Achanta, F.J. Estrada, P. Wils, S. Süsstrunk, Salient region detection and segmentation, in Computer Vision Systems (Springer, Berlin, 2008), pp. 66–75

  6. B. Alexe, T. Deselaers, V. Ferrari, Measuring the objectness of image windows. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2189–2202 (2012)

    Article  Google Scholar 

  7. A. Borji, L. Itti, Exploiting local and global patch rarities for saliency detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2012), pp. 478–485

  8. C. Christopoulos, A. Skodras, T. Ebrahimi, The JPEG2000 still image coding system:an overview. IEEE Trans. Consum. Electron. 46(4), 1103–1127 (2000)

    Article  Google Scholar 

  9. M.-M. Cheng, G.-X. Zhang, N. J. Mitra, X. Huang, S.-M. Hu, Global contrast based salient region detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2011), pp. 409–416

  10. L. Duan, C. Wu, J. Miao, L. Qing, Y. Fu, Visual saliency detection by spatially weighted dissimilarity, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2011), pp. 473–480

  11. Y.M. Fang, Z.Z. Chen, W.S. Lin, C.W. Lin, Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans. Image Process. 21(9), 3888–3901 (2012)

    Article  MathSciNet  Google Scholar 

  12. J. Feng, Y.C. Wei, L.T. Tao, C. Zhang, J. Sun, Salient object detection by composition,in Proceedings of the IEEE International Conference on Computer Vision, (2011), pp. 1028–1035

  13. K. Guo, P. Ishwar, J. Konrad, Action recognition using sparse representation on covariance manifolds of optical flow, in The 7th IEEE International Conference on Advanced Video and Signal Based Surveillance, (2010), pp. 188–195

  14. S. Gao, I.W.H. Tsang, L.T. Chia, Sparse representation with kernels. IEEE Trans. Image Process. 22(2), 423–434 (2013)

    Article  MathSciNet  Google Scholar 

  15. S. Goferman, L. Zelnik-Manor, A. Tal, Context-aware saliency detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2010), pp. 2376–2383

  16. P.S. Hiremath, J. Pujari, Content based image retrieval using color boosted salient points and shape features of an image. Int. J. Image Process. 2(1), 10–17 (2008)

    Google Scholar 

  17. J. Harel, C. Koch, P. Perona, Graph-based visual saliency, in Advances in Neural Processing Systems, (2006), pp. 545–552

  18. X.D. Hou, L.Q. Zhang, Saliency detection: a spectral residual approach, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2007), pp. 1–8

  19. M.T. Harandi, C. Sanderson, R. Hartley, B. C. Lovell, Sparse coding and dictionary learning for symmetric positive definite matrices: a kernel approach, in Proceedings of the11th European Conference on Computer Vision, (2012), pp. 216–229

  20. L. Itti, C. Koch, E. Niebur, A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  21. M.W. Jian, J.Y. Dong, J. Ma, Image retrieval using wavelet-based salient regions. Imaging Sci. J. 59(4), 219–231 (2011)

    Article  Google Scholar 

  22. T. Judd, K. Ehinger, F. Durand, A. Torralba, Learning to predict where humans look, in Proceedings of the IEEE International Conference on Computer Vision, (2009), pp. 2106–2113

  23. A.D. Klein, S. Frintrop, Center-surround divergence of feature statistics for salient object detection, in Proceedings of the IEEE International Conference on Computer Vision, (2011), pp. 2214–2219

  24. T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, H.-Y. Shum, Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 353–367 (2011)

    Article  Google Scholar 

  25. P. Li, Q. Wang, W. Zuo, L. Zhang, Log-Euclidean Kernels for sparse representation and dictionary learning, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2013), pp. 1601–1608

  26. X. Li, H. Lu, L. Zhang, X. Ruan, M.-H. Yang, Saliency detection via dense and sparse reconstruction, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2013), pp. 2976–2983

  27. Y. Ma, B. Miller, I. Cohen, Video sequence querying using clustering of objects’ appearance models, in Advances in Visual Computing, (2007), pp. 328–329

  28. Y.-F. Ma, H.J. Zhang, Contrast-based image attention analysis by using fuzzy growing, in Proceedings of the 11th ACM International Conference on Multimedia, (2003), pp. 374–381

  29. F. Perazz, P. Krahenbuhl, Y. Pritch, A. Hornung, Saliency filters: contrast based filtering for salient region detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2012), pp. 733–740

  30. F. Porikli, O. Tuzel, P. Meer, Covariance tracking using model update based on lie algebra, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2006), pp. 728–735

  31. Y. Pang, Y. Yuan, X. Li, Gabor-based region covariance matrices for face recognition. IEEE Trans. Circuits Syst. Video Technol. 18(7), 989–993 (2008)

    Article  Google Scholar 

  32. X. Pennec, P. Fillard, N. Ayache, Riemannian framework for tensor computing. Int. J. Comput. Vis. 66(1), 41–66 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  33. U. Rutishause, D. Walther, C. Koch, P. Perona, Is bottom-up attention useful for object recognition?, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2012), pp. 2–37

  34. E. Rahtu, J. Kannala, M. Salo, J. Heikkilä, Segmenting salient objects from images and videos, in Proceedings of the11th European Conference on Computer Vision, (2010), pp. 366–379

  35. G. Sharma, F. Jurie, C. Schmid, Discriminative spatial saliency for image classification, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2012), pp. 3506–3513

  36. X. Shen, Y. Wu, A unified approach to salient object detection via low rank matrix recovery, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2012), pp. 853–860

  37. R. Sivalingam, D. Boley, V. Morellas, V.N. Papanikolopoulos, Tensor sparse coding for region covariances, in Proceedings of the11th European Conference on Computer Vision, (2010), pp. 722–735

  38. A. Treisman, G. Gelade, A feature-integration theory of attention. Cognit Psychol 12(1), 97–136 (1980)

    Article  Google Scholar 

  39. O. Tuzel, F. Porikl, P. Meer, Region covariance: a fast descriptor for detection and classification, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2006), pp. 589–600

  40. O. Tuzel, F. Porikl, P. Meer, Pedestrian detection via classification on Riemannian manifolds. IEEE Trans. Pattern Anal. Mach. Intell. 30(10), 1713–1727 (2008)

    Article  Google Scholar 

  41. O. Tuze, F. Porikli, P. Meer, Human detection via classification on Riemannian manifolds, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2007), pp. 1–8

  42. H.W. Tian, Y.M. Fang, Y. Zhao, W.S. Lin, R.R. Ni, Z.F. Zhu, Salient region detection by fusing bottom-up and top-down features extracted from a single image. IEEE Trans. Image Process. 23(10), 4389–4398 (2014)

    Article  MathSciNet  Google Scholar 

  43. Y. Wei, F. Wen, W. Zhu, J. Sun, Geodesic saliency using background priors, in Proceedings of the11th European Conference on Computer Vision, (2012), pp. 29–42

  44. Y.L. Xie, H.C. Lu, Visual saliency detection based on Bayesian model, in 18th IEEE International Conference on Image Processing, (2011), pp. 645–648

  45. Y. Xie, H. Lu, M.-H. Yang, Bayesian saliency via low and mid level cues. IEEE Trans. Image Process. 22(5), 1689–1698 (2013)

    Article  MathSciNet  Google Scholar 

  46. Y. Xie, B. C. Vemuri, J. Ho, Statistical analysis of tensor fields, in Medical Image Computing and Computer-Assisted Intervention, (2010), pp. 682–689

  47. Y. Zhai, M. Shah, Visual attention detection in video sequences using spatiotemporal cues, in Proceedings of the 14th Annual ACM International Conference on Multimedia, (2006), pp. 815–824

  48. Z.-F. Zhu, Q. Chen, Y. Zhao, Ensemble dictionary learning for saliency detection. Image Vis. Comput. 32(3), 180–188 (2014)

    Article  Google Scholar 

Download references

Acknowledgments

The work described in this paper is supported by the Science and Technology Foundation of Henan Province of China (Grant No. 142102310562).

Author information

Authors and Affiliations

  1. Physics and Electronic Engineering College, Nanyang Normal University, Nanyang, 473061, Henan, People’s Republic of China

    Ying-Ying Zhang, Zhi-Peng Wang & Xiao-Dong Lv

  2. Oil Equipment Intelligent Control Engineering Laboratory of Henan Province, Nanyang, 473061, Henan, People’s Republic of China

    Ying-Ying Zhang

Authors
  1. Ying-Ying Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zhi-Peng Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xiao-Dong Lv
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ying-Ying Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, YY., Wang, ZP. & Lv, XD. Saliency Detection via Sparse Reconstruction Errors of Covariance Descriptors on Riemannian Manifolds. Circuits Syst Signal Process 35, 4372–4389 (2016). https://doi.org/10.1007/s00034-016-0267-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-016-0267-x

Keywords