Skip to main content
SpringerLink
Log in

Point-wise saliency detection on 3D point clouds via covariance descriptors

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In the human visual system, visual saliency perception is a rapid pre-attention processing mechanism, which can benefit myriad visual tasks, such as segmentation, localization, and detection. While most research is devoted to saliency detection on 2D images and 3D meshes, little work has been performed for efficient saliency detection on 3D point clouds. In this paper, we present a novel point clouds saliency detection method by employing principal component analysis (PCA) in a sigma-set feature space. In this method, we first construct local shape descriptors based on covariance matrices for saliency detection, considering that covariance matrices can naturally model nonlinear correlations of different low-level compact and rotational-invariant features. Secondly, we transform these covariance matrices to vector descriptors in Euclidean vector space by applying the sigma-point technique, which keeps the inner statistics of regions of 3D point clouds. Based on our informative descriptors, PCA is employed in the descriptor space for identifying saliency patterns in a point cloud. Our method shows its advantages of being structure-sensitive, capturing geometry information and computationally efficient. Experimental results demonstrate that our method achieves good performance without using any topological information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Akman, O., Jonker, P.: Computing saliency map from spatial information in point cloud data. In: Proceedings of Advanced Concepts for Intelligent Vision Systems—International Conference, Acivs 2010, Sydney, December 13–16, 2010, pp. 290–299 (2010)

  2. Castellani, U., Cristani, M., Fantoni, S., Murino, V.: Sparse points matching by combining 3D mesh saliency with statistical descriptors. Comput. Graph. Forum 27(2), 643–652 (2008)

    Article  Google Scholar 

  3. Chen, X., Saparov, A., Pang, B., Funkhouser, T.: Schelling points on 3D surface meshes. ACM Trans. Graph. 31(4), 13–15 (2012)

    Article  Google Scholar 

  4. Cheng, M., Zhang, G., Mitra, N.J., Huang, X., Hu, S.: Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 409–416 (2015)

    Article  Google Scholar 

  5. Cirujeda, P., Dicente Cid, Y., Mateo, X., Binefa, X.: A 3D scene registration method via covariance descriptors and an evolutionary stable strategy game theory solver. Int. J. Comput. Vis. 115(3), 306–329 (2015)

    Article  MathSciNet  Google Scholar 

  6. Desbrun, M., Meyer, M., Schröder, P., Barr, A.H.: Implicit fairing of irregular meshes using diffusion and curvature flow. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’99, pp. 317–324 (1999)

  7. Dutagaci, H., Cheung, C.P., Godil, A.: Evaluation of 3D interest point detection techniques via human-generated ground truth. Vis. Comput. 28(9), 901–917 (2012)

    Article  Google Scholar 

  8. Förstner, W., Moonen, B.: A Metric for Covariance Matrices. Springer, Berlin (2003)

    Book  Google Scholar 

  9. Godil, A., Wagan, A.I.: Salient local 3D features for 3D shape retrieval. In: 3D image processing (3DIP) and applications II. SPIE, Bellingham (2011)

  10. Guy, G., Medioni, G.: Inference of surfaces, 3-D curves, and junctions from sparse 3-D data. In: Proceedings of the International Symposium on 1995, Computer Vision, pp. 599–604 (1995)

  11. Hariri, W., Tabia, H., Farah, N., Benouareth, A., Declercq, D.: 3D face recognition using covariance based descriptors. Pattern Recognit. Lett. 78, 1–7 (2016)

    Article  Google Scholar 

  12. Hong, X., Chang, H., Shan, S., Chen, X., Gao, W.: Sigma set: a small second order statistical region descriptor. In: Computer Vision and Pattern Recognition, IEEE Conference on 2009. CVPR 2009, pp. 1802–1809 (2009)

  13. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

  14. Howlett, S., Hamill, J., O’Sullivan, C.: Predicting and evaluating saliency for simplified polygonal models. ACM Trans. Appl. Percept. 2(3), 286–308 (2005)

    Article  Google Scholar 

  15. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  16. Le Meur, O., Baccino, T.: Methods for comparing scanpaths and saliency maps: strengths and weaknesses. Behav. Res. Methods 45(1), 251–266 (2013)

    Article  Google Scholar 

  17. Lee, C.H., Varshney, A., Jacobs, D.W.: Mesh saliency. ACM Trans. Graph. 24(3), 659–666 (2005)

    Article  Google Scholar 

  18. Leifman, G., Shtrom, E., Tal, A.: Surface regions of interest for viewpoint selection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 414–421 (2012)

  19. Liu, X., Liu, L., Song, W., Liu, Y., Ma, L.: Shape context based mesh saliency detection and its applications: a survey. Comput. Graph. 57, 12–30 (2016)

    Article  Google Scholar 

  20. Liu, X., Tao, P., Cao, J., Chen, H., Zou, C.: Mesh saliency detection via double absorbing Markov chain in feature space. Vis. Comput. 32(9), 1121–1132 (2016)

    Article  Google Scholar 

  21. Nealen, A., Igarashi, T., Sorkine, O., Alexa, M.: Laplacian mesh optimization. In: Proceedings of the 4th International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia, GRAPHITE ’06, pp. 381–389 (2006)

  22. Novatnack, J., Nishino, K.: Scale-dependent 3D geometric features. In: IEEE 11th International Conference on Computer Vision 2007, pp. 1–8 (2007)

  23. Pauly, M., Gross, M., Kobbelt, L.P.: Efficient simplification of point-sampled surfaces. In: IEEE Visualization, 2002. VIS 2002, pp. 163–170 (2002)

  24. Pauly, M., Keiser, R., Gross, M.: Multi-scale feature extraction on point-sampled surfaces. In: Computer Graphics Forum, pp. 281–289 (2003)

  25. Qi, W., Han, J., Zhang, Y., Bai, L.: Saliency detection via boolean and foreground in a dynamic Bayesian framework. Vis. Comput. 33(2), 209–220 (2017)

    Article  Google Scholar 

  26. Ran, G., Cohen-Or, D.: Salient geometric features for partial shape matching and similarity. ACM Trans. Graph. 25(1), 130–150 (2006)

    Article  Google Scholar 

  27. Ran, M., Tal, A., Zelnikmanor, L.: What makes a patch distinct? In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1139–1146 (2013)

  28. Rijsbergen, C.J.V.: Information Retrieval, 2nd edn. Butterworth-Heinemann, Newton (1979)

    MATH  Google Scholar 

  29. Rusu, R.B.: Semantic 3D object maps for everyday manipulation in human living environments. Ph.D. thesis, Computer Science Department, Technische Universitaet Muenchen, Germany (2009)

  30. Shilane, P., Funkhouser, T.: Distinctive regions of 3D surfaces. ACM Trans. Graph. 26(2), 2007 (2007)

  31. Shtrom, E., Leifman, G., Tal, A.: Saliency detection in large point sets. In: IEEE International Conference on Computer Vision 2013, pp. 3591–3598 (2013)

  32. Sipiran, I., Bustos, B.: Harris 3D: a robust extension of the harris operator for interest point detection on 3D meshes. Vis. Comput. 27(11), 963–976 (2011)

    Article  Google Scholar 

  33. Song, R., Liu, Y., Martin, R.R., Echavarria, K.R.: Local-to-global mesh saliency. Vis. Comput. (2016). doi:10.1007/s00371-016-1334-9

  34. Song, R., Liu, Y., Martin, R.R., Rosin, P.L.: Mesh saliency via spectral processing. ACM Trans. Graph. 33(1), 57–76 (2014)

    Article  MATH  Google Scholar 

  35. Stas, G., Lihi, Z.M., Ayellet, T.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)

    Article  Google Scholar 

  36. Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. In: Proceedings of the Symposium on Geometry Processing, SGP ’09, pp. 1383–1392 (2009)

  37. Tabia, H., Laga, H.: Covariance-based descriptors for efficient 3D shape matching, retrieval, and classification. IEEE Trans. Multimed. 17(9), 1591–1603 (2015)

    Article  Google Scholar 

  38. Tao, P., Cao, J., Li, S., Liu, L., Liu, X.: Mesh saliency via ranking unsalient patches in a descriptor space. Comput. Graph. 46, 264–274 (2014)

    Article  Google Scholar 

  39. Tasse, F.P., Kosinka, J., Dodgson, N.A.: Cluster-based point set saliency. In: IEEE International Conference on Computer Vision, pp. 637–639 (2015)

  40. Tasse, F.P., Kosinka, J., Dodgson, N.A.: Quantitative analysis of saliency models. In: SIGGRAPH ASIA 2016 Technical Briefs, SA ’16, ACM, pp. 19:1–19:4 (2016)

  41. Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: Proceedings of Computer Vision—ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7–13, 2006, Part II, pp. 589–600. Springer, Berlin (2006)

  42. Tuzel, O., Porikli, F., Meer, P.: Pedestrian detection via classification on riemannian manifolds. IEEE Trans. Pattern Anal. Mach. Intell. 30(10), 1713–1727 (2008)

    Article  Google Scholar 

  43. Wang, S., Li, N., Li, S., Luo, Z., Su, Z., Qin, H.: Multi-scale mesh saliency based on low-rank and sparse analysis in shape feature space. Comput. Aided Geom. Des. 35–36, 206–214 (2015)

    Article  MathSciNet  Google Scholar 

  44. Wang, Y., Liu, R., Song, X., Su, Z.: A nonlocal \(L_0\) model with regression predictor for saliency detection and extension. Vis. Comput., 1–16 (2016)

  45. Wu, J., Shen, X., Zhu, W., Liu, L.: Mesh saliency with global rarity. Graph. Models 75(5), 255–264 (2013)

    Article  Google Scholar 

  46. Yang, B., Li, F.W.B., Wang, X., Xu, M., Liang, X., Jiang, Z., Jiang, Y.: Visual saliency guided textured model simplification. Vis. Comput. 32(11), 1415–1432 (2016)

    Article  Google Scholar 

  47. Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: Computer Vision and Pattern Recognition (CVPR), IEEE Conference on 2013, pp. 3166–3173 (2013)

  48. Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency optimization from robust background detection. In: Computer Vision and Pattern Recognition, pp. 2814–2821 (2014)

Download references

Acknowledgements

The authors gratefully acknowledge the anonymous reviewers for their comments to help us to improve our paper and also thank for their enormous help in revising this paper. This work was partially supported by the National Natural Science Foundation of China under Grant No. 61231018 and the Programme of Introducing Talents of Discipline to Universities under Grant B13043.

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049, China

    Yu Guo,  Fei Wang & Jingmin Xin

  2. The National Engineering Laboratory for Visual Information Processing and Applications, Xi’an Jiaotong University, Xi’an, 710049, China

    Yu Guo,  Fei Wang & Jingmin Xin

  3. Shaanxi Provincial Key Laboratory of Digital Technology and Intelligent Systems, Xi’an, 710049, China

    Yu Guo,  Fei Wang & Jingmin Xin

Authors
  1. Yu Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingmin Xin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fei Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, Y., Wang, . & Xin, J. Point-wise saliency detection on 3D point clouds via covariance descriptors. Vis Comput 34, 1325–1338 (2018). https://doi.org/10.1007/s00371-017-1416-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-017-1416-3

Keywords

Search

Navigation