Skip to main content
SpringerLink
Log in

Discriminative geodesic Gaussian process latent variable model for structure preserving dimension reduction in clustering and classification problems

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Dimension reduction is a common approach for analyzing complex high-dimensional data and allows efficient implementation of classification and decision algorithms. Gaussian process latent variable model (GPLVM) is a widely applicable dimension reduction method which represents latent space without considering the class labels. Preserving the structure and topology of data are key factors that influence the performance of dimensionality reduction models. A conventional measure which reflects the topological structure of data points is geodesic distance. In this study, we propose an enriched GPLVM mapping between low-dimensional space and high-dimensional data. One of the contributions of the proposed approach is to calculate geodesic distance under the influence of class labels and introducing an improved GPLVM kernel using the distance. Also, the objective function of the model is reformulated to consider the trade-off between class separation and structure preservation which improves discrimination power and compactness of data. The efficiency of the proposed approach is compared with other dimension reduction techniques such as the kernel principal component analysis (KPCA), locally linear embedding (LLE), Laplacian eigenmaps and also discriminative and supervised extensions of standard GPLVM. Based on the experiments, it is suggested that the proposed model has a higher capacity for accurate classification and clustering of data as compared with the mentioned approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  2. Pudil P, Novovičová J (1998) Novel methods for feature subset selection with respect to problem knowledge. Feature extraction, construction and selection. Springer, Berlin, pp 101–116

    Chapter  Google Scholar 

  3. Tipping ME, Bishop CM (1999) Probabilistic principal component analysis. J R Stat Soc Ser B (Stat Methodol) 61(3):611–622

    Article  MathSciNet  MATH  Google Scholar 

  4. Agapito L, Bronstein MM, Rother C (2015) Computer vision—ECCV 2014 workshops, Zurich, Switzerland, September 6–7 and 12, 2014, Proceedings, no. pt. 4. Springer

  5. Schölkopf B, Smola A, Müller K-R (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319

    Article  Google Scholar 

  6. Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. NIPS 14:585–591

    Google Scholar 

  7. Li H, Teng L, Chen W, Shen IF (2005) Supervised learning on local tangent space. In: Wang J, Liao X, Yi Z (eds) Advances in neural networks—ISNN 2005, vol 3496. Springer, Berlin, pp 546–551

    Chapter  Google Scholar 

  8. Lawrence ND (2006) The Gaussian process latent variable model. The University of Sheffield, Department of Computer Science, Technical Report

  9. Jiang X, Gao J, Wang T, Zheng L (2012) Supervised latent linear Gaussian process latent variable model for dimensionality reduction. IEEE Trans Syst Man Cybern Part B Cybern 42(6):1620–1632

    Article  Google Scholar 

  10. Gao X, Wang X, Tao D, Li X (2011) Supervised Gaussian process latent variable model for dimensionality reduction. IEEE Trans Syst Man Cybern Part B Cybern 41(2):425–434

    Article  Google Scholar 

  11. Romero J, Feix T, Ek CH, Kjellstrom H, Kragic D (2013) Extracting postural synergies for robotic grasping. IEEE Trans Robot 29(6):1342–1352

    Article  Google Scholar 

  12. Han L, Wu X, Liang W, Hou G, Jia Y (2010) Discriminative human action recognition in the learned hierarchical manifold space. Image Vis Comput 28(5):836–849

    Article  Google Scholar 

  13. Serradilla J, Shi JQ, Morris AJ (2011) Fault detection based on Gaussian process latent variable models. Chemom Intell Lab Syst 109(1):9–21

    Article  Google Scholar 

  14. Lawrence N (2005) Probabilistic non-linear principal component analysis with Gaussian process latent variable models. J Mach Learn Res 6:1783–1816

    MathSciNet  MATH  Google Scholar 

  15. Urtasun R, Darrell T (2007) Discriminative Gaussian process latent variable model for classification. In: Proceedings of the 24th international conference on machine learning. ACM, Corvalis, Oregon, pp 927–934

  16. Ek CH, Jaeckel P, Campbell N, Lawrence ND, Melhuish C (2009) Shared Gaussian process latent variable models for handling ambiguous facial expressions. Am Inst Phys Conf Ser 1107:147–153

    Google Scholar 

  17. Ek CH, Lawrence PHSTND (2009) Shared Gaussian process latent variable models. Ph.D. thesis

  18. Wang X, Gao X, Yuan Y, Tao D, Li J (2010) Semi-supervised Gaussian process latent variable model with pairwise constraints. Neurocomputing 73(10):2186–2195

    Article  Google Scholar 

  19. Hensman J, Fusi N, Lawrence ND (2013) Gaussian processes for big data. CoRR, vol. abs/1309.6835

  20. van der Maaten LJP, Postma EO, van den Herik HJ (2009) Dimensionality reduction: a comparative review. J Mach Learn Res 10(1–41):66–71

    Google Scholar 

  21. Chun-Guang L, Jun G (2006) Supervised Isomap with explicit mapping. In: First international conference on innovative computing, information and control, 2006. ICICIC’06, vol 3, pp 345–348

  22. Kaski S, Peltonen J (2011) Dimensionality reduction for data visualization [applications corner]. IEEE Signal Process Mag 28(2):100–104

    Article  Google Scholar 

  23. Venna J, Kaski S (2007) Nonlinear dimensionality reduction as information retrieval. In: Artificial intelligence and statistics, pp 572–579

  24. Gorban AN, Kégl B, Wunsch DC, Zinovyev A (2008) Principal manifolds for data visualization and dimension reduction, vol 58. Springer, Berlin

    Book  MATH  Google Scholar 

  25. Kaski S, Nikkilä J, Oja M, Venna J, Törönen P, Castrén E (2003) Trustworthiness and metrics in visualizing similarity of gene expression. BMC Bioinform 4(1):48

    Article  Google Scholar 

  26. Chen J, Liu Y (2011) Locally linear embedding: a survey. Artif Intell Rev 36(1):29–48

    Article  Google Scholar 

  27. Kovács F, Legány C, Babos A (2005) Cluster validity measurement techniques. In: 6th International symposium of hungarian researchers on computational intelligence

  28. Wagner S, Wagner D (2007) Comparing clusterings: an overview. Universität Karlsruhe, Fakultät für Informatik, Karlsruhe

    Google Scholar 

  29. Rand WM (1971) Objective criteria for the evaluation of clustering methods. J Am Stat Assoc 66(336):846–850

    Article  Google Scholar 

  30. Hubert L, Arabie P (1985) Comparing partitions. J Classif 2(1):193–218

    Article  MATH  Google Scholar 

  31. Meilă M (2007) Comparing clusterings—an information based distance. J Multivar Anal 98(5):873–895

    Article  MathSciNet  MATH  Google Scholar 

  32. Fowlkes EB, Mallows CL (1983) A method for comparing two hierarchical clusterings. J Am Stat Assoc 78(383):553–569

    Article  MATH  Google Scholar 

  33. Lingras P, Haider F, Triff M (2016) Granular meta-clustering based on hierarchical, network, and temporal connections. Granul Comput 1(1):71–92

    Article  Google Scholar 

  34. Peters G, Weber R (2016) DCC: a framework for dynamic granular clustering. Granul Comput 1(1):1–11

    Article  Google Scholar 

  35. Antonelli M, Ducange P, Lazzerini B, Marcelloni F (2016) Multi-objective evolutionary design of granular rule-based classifiers. Granul Comput 1(1):37–58

    Article  Google Scholar 

  36. Liu H, Cocea M (2017) Granular computing-based approach for classification towards reduction of bias in ensemble learning. Granul Comput 2(3):131–139

    Article  Google Scholar 

  37. Livi L, Sadeghian A (2016) Granular computing, computational intelligence, and the analysis of non-geometric input spaces. Granul Comput 1(1):13–20

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Software Engineering, Mashhad Branch, Islamic Azad University, Mashhad, Iran

    Mahdi Heidari & Mohammad Hossein Moattar

Authors
  1. Mahdi Heidari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Hossein Moattar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Hossein Moattar.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest in the manuscript.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Heidari, M., Moattar, M.H. Discriminative geodesic Gaussian process latent variable model for structure preserving dimension reduction in clustering and classification problems. Neural Comput & Applic 31, 3265–3278 (2019). https://doi.org/10.1007/s00521-017-3273-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-017-3273-4

Keywords

Search

Navigation