Skip to main content
SpringerLink
Log in

An unsupervised multi-manifold discriminant isomap algorithm based on the pairwise constraints

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In this paper, an unsupervised multi-manifold Isomap algorithm, which is named UMD-Isomap, is proposed for the purpose of dimensionality reduction and clustering of multi-manifold data. First, the global pairwise constraints are constructed by training m mixtures of probabilistic principal component analyzers (MPPCA) and propagating their local tangent subspaces. At the same time, the sub-manifolds are also clustered, and their classes information are recorded in the pairwise constraints. If the number of sub-manifolds is known, a new pairwise constraints is computed by using a cluster ensemble algorithm, which creates a similarity matrix by accumulating c sets of pairwise constraints. Subsequently, a new objective function with pairwise constraints and two supervised solutions are proposed to achieve the dimensionality reduction of the multi-manifolds. The proposed UMD-Isomap algorithm achieved better performance in terms of dimensionality reduction and clustering accuracy than other commonly used methods and its effectiveness was verified.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. The experiment is carried out with the python version of Umap algorithm, and its results in figures are displayed by MATLAB.

References

  1. Tenenbaum J, Silva DD, Langford J (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Article  Google Scholar 

  2. Roweis S, Saul L (2000) Nonlinear dimentionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  3. Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396

    Article  Google Scholar 

  4. Lin T, Zha HB (2008) Riemannian manifold learning. IEEE Trans. Pattern Anal. Mach. Intell. 30(5):796–809

    Article  Google Scholar 

  5. Liu Y, Nie F et al (2019) Flexible unsupervised feature extraction for image classification. Neural Netw. 115:65–71

    Article  Google Scholar 

  6. Liu Y, Gao Q, Yang Z et al (2018) Learning with Adaptive Neighbors for Image Clustering. In: Twenty-Seventh International Joint Conference on Artificial Intelligence IJCAI-18. 2018, 2483-2489

  7. Fan B, Kong QQ et al (2020) Efficient nearest neighbor search in high dimensional hamming space. pattern recognition. https://doi.org/10.1016/j.patcog.2019.107082

  8. Zheng Y, Liu X, Chen S et al (2020) Multi-task deep dual correlation filters for visual tracking. IEEE Trans. Image Process. 29:9614–9626

    Article  Google Scholar 

  9. Zheng YH, Jeon B et al (2018) Student’s t-Hidden Markov Model for Unsupervised Learning Using Locallized Feature Selection. IEEE Transactions on Circuits and Systems for Video Technology 28(10):2586–2598

  10. Souvenir R, Manifold Pless R (2005) Clustering. Proceedings 1(1):648–653

    Google Scholar 

  11. Meng D, Leung Y, Fung T, Xu Z (2008) Nonlinear dimensionality reduction of data lying on the multicluster manifold. IEEE Trans Syst Man Cybernet-Part B 38(4):1111–1122

    Article  Google Scholar 

  12. Gao XF, Liang JY (2013) Manifold learning algorithm DC-ISOMAP of data lying on well-separated multi-manifold with same intrinsic dimension. J Comput Res Dev 50(8):1690–1699

    Google Scholar 

  13. Ng A, Jordan M, Weiss Y (2001) On spectral clustering: Analysis and an algorithm. in Advances in Neural Information Processing Systems 14. 2001: 849-856

  14. Shi JB, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  15. Zhang K, Kwok JT (2010) Clustered Nyström method for large scale manifold learning and dimension reduction. IEEE Trans Neural Netw 21(10):1576–1587

    Article  Google Scholar 

  16. Wang Y, Jiang Y, Zhou ZH (2011) Spectral clustering on multiple manifolds. IEEE Trans Neural Netw 22(7):1149–1161

    Article  Google Scholar 

  17. Wang Y, Jiang Y, Wu Y, et al (2011) Local and Structural Consistency for Multi-manifold Clustering. Proc of IJCAI 2011. Menlo Park, CA: AAAI Press, 1559-1564

  18. Babaeian A, Bayestehtashk A, Bandarabadi M. Multiple Manifold Clustering Using Curvature Constrained Path. PLoS ONE 10(9): e0137986. https://doi.org/10.1371/journal.pone.0137986

  19. van der Maaten L, Hinton GE (2008) Visualizing high-dimensional data using t-SNE. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

  20. van der Maaten L (2014) Accelerating t-SNE using Tree-Based Algorithms. Journal of Machine Learning Research 15(Oct):3221-3245

  21. McInnes L, Healy J, Melville J (2018) UMAP: uniform manifold approximation and projection for dimension reduction. J Open Source Softw 3(29):861

    Article  Google Scholar 

  22. Basu S, Davidson I, Wagstaff K (2008) Constrained clustering: advanced in algorithms, theory, and applications. CRC Press, Boca Raton

    Book  Google Scholar 

  23. Geng X, Dc Zhan, Zhou ZH (2005) supervised nonlinear dimensionality reduction for visualization and classification. IEEE Trans Syst Man Cybernet-Part B 35(6):1098–1107

    Article  Google Scholar 

  24. Goldberg A, Zhu X et al (2009) Multi-manifold semi-supervised learning. In: Proceedings of the 12th International Conference on Artificial Intelligence and Statistics, 2009, 169-176

  25. Zhang Z, Chow TW, Zhao M (2013) M-Isomap:orthogonal constrained marginal Isomap for nonlinear dimensionality reduction. IEEE Trans. Cybern 43(1):180–191

    Article  Google Scholar 

  26. Yang B, Xiang M, Zhang Y (2016) Multi-manifold Discriminant Isomap for visualization and classification. Pattern Recognit 2016(55):215–230

    Article  Google Scholar 

  27. Zhang Y, Zhang Z et al (2017) Semi-supervised local multi-manifold Isomap by linear embedding for feature extraction. Pattern Recoginit 2017(000):1–17

    Google Scholar 

  28. Fan B, Liu HM et al (2020) Deep Unsupervised Binary Descriptor Learning through Locality Consistency and Self Distinctiveness. IEEE Transactions on Multimedia. https://doi.org/10.1109/TMM.2020.3016122

  29. Tipping ME, Bishop CM (1999) Mixtures of probabilistic principal component analysers. Neural Comput. 11(2):443–482

    Article  Google Scholar 

  30. Zhao J (2014) Efficient Mofel selection for mixtures of probabilistic PCA via hierarchical BIC. IEEE Trans Cybernet 44(10):1871–1883

    Article  Google Scholar 

  31. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B (Methodological) 39(1):1–38

    MathSciNet  MATH  Google Scholar 

  32. Sun D, Zhang D (2010) Bagging constraint score for feature selection with pairwise constraints. Pattern Recognit 43(6):2106–2118

    Article  Google Scholar 

  33. Baghshah M S , Shouraki S B. Semi-supervised metric learning using pairwise constraints. In: Proceedings of the International Joint Conference on Artificial Intelligence, 2009, 9: 1217-1222

  34. Wang F (2011) Semi-supervised metric learning by maximizing constraint margin. IEEE Trans Syst, Man Cybernet Part B-Cybernet 41(4):931–939

    Article  Google Scholar 

  35. Joint Optimization for Pairwise Constraint Propagation, IEEE TNNLS, 2020

  36. (2020) Pairwise Constraint Propagation With Dual Adversarial Manifold Regularization. IEEE TNNLS

  37. Leeuw J de. Applications of Convex Analysis to Multidimensional Scaling. In JR Barra, F Brodeau, G Romier, B van Cutsem (eds.), Recent Developments in Statistics (1977) 133–145. North Holland Publishing Company, Amsterdam

  38. de Leeuw I, Mair P (2009) Multidimensional scaling using majorization: SMACOF in R. J Stat Softw 31(3):1–30

    Article  Google Scholar 

  39. Saul L, Roweis S (2004) Think globally, fit locally: unsupervised learning of low dimensional manifolds. J Mach Learn Res 4(2):119–155

    MathSciNet  MATH  Google Scholar 

  40. Donoho D, Grimes C (2003) Hessian eigenmaps: locally linear embedding techniques for high-dimensional data. PNAS. 100(10):5591–5596

    Article  MathSciNet  Google Scholar 

  41. Zhang ZY, Zha HY (2005) Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J Sci Comput 26(1):313–338

    Article  MathSciNet  Google Scholar 

  42. Gao XF, Liang JY (2011) The dynamical neighborhood selection based on the sampling density and manifold curvature for isometric data embedding. Pattern Recognit Lett 32:202–209

    Article  Google Scholar 

  43. Steyvers M (2002) Multidimensional scaling, In Encyclopedia of. Cognitive Science

Download references

Acknowledgements

This work is partially supported by the National Natural Science Foundation of China (No. 61703252); Applied Basic Research Programs of Shanxi Province (201701D121053) and Research Project Supported by Shanxi Scholarship Council of China (2016-002).

Author information

Authors and Affiliations

  1. College of Computer and Information Technology, Shanxi University, Taiyuan, China

    Xiaofang Gao, Jiye Liang, Wenjian Wang, Xuefei Bai & Lina Jia

  2. Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Taiyuan, China

    Jiye Liang & Wenjian Wang

Authors
  1. Xiaofang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiye Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenjian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuefei Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lina Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaofang Gao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, X., Liang, J., Wang, W. et al. An unsupervised multi-manifold discriminant isomap algorithm based on the pairwise constraints. Int. J. Mach. Learn. & Cyber. 13, 1317–1336 (2022). https://doi.org/10.1007/s13042-021-01449-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01449-8

Keywords

Search

Navigation