Skip to main content
SpringerLink
Log in

Supervised hierarchical neighborhood graph construction for manifold learning

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Neighborhood graph construction plays a vital role in the success of the manifold learning. This paper introduces a supervised hierarchical neighborhood graph construction method for manifold learning. The goal of our method is to discriminate among profoundly close data points with different classes. Our proposed method includes two main phases. The first phase consists of a supervised form of neighborhood selection. The first phase aims to incorporate the Euclidean distance between two objects and their class labels for providing a reasonable estimate of the data’s local topology. The second phase optimizes the neighborhood graph based on the proposed hierarchical search method to find transitive similarity between objects. The second phase aims to improve the representation of the global topology of the manifold. Our proposed method defines a function that compares two data points in a semantically relevant way. The proposed method helps to identify the right neighborhood of each object, and thus, the resulting graph best approximates the actual structure of the manifold. In the experiments, we used seven state-of-the-art works on challenging synthetic and real-world data sets to demonstrate the superiority of the proposed method. In face recognition, our neighborhood graph can improve the canonical LLE and Isomap more than 20 and 50%, respectively. Also, LLE and Isomap based on our proposed method achieve more than 24 and 16% better results respect to the best result of state-of-the-art graph-based manifold learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Available at: https://lvdmaaten.github.io/drtoolbox/.

References

  1. Zheng, H., Horace, H.S.Ip: Image classification and annotation based on robust regularized coding. SIVP 10(1), 55–64 (2016)

    Google Scholar 

  2. Wei, Z., Chu, Y., Zhao, L.: An effective two-dimensional linear discriminant analysis with locality preserving approach for image recognition. SIVP 11(8), 1577–1584 (2017)

    Google Scholar 

  3. Yildiz, K.: Dimensionality reduction-based feature extraction and classification on fleece fabric images. SIVP 11(2), 317–323 (2016)

    MathSciNet  Google Scholar 

  4. Vretos, N., Tefas, A., Pitas, I.: A novel dimensionality reduction technique based on kernel optimization through graph embedding. SIVP 9(1), 3–10 (2015)

    Google Scholar 

  5. Dahmane, A., Larabi, S., Bilasco, L.M., Djeraba, C.: Head pose estimation based on face symmetry analysis. SIVP 9(8), 1871–1880 (2015)

    Google Scholar 

  6. Lu, J., Tan, Y.-P., Wang, G.: Discriminative multimanifold analysis for face recognition from a single training sample per person. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 39–51 (2013)

    Article  Google Scholar 

  7. Campanella, M., Leonardi, R., Migliorati, P.: Interactive visualization of video content and associated description for semantic annotation. SVIP 3(2), 183–196 (2009)

    Google Scholar 

  8. Berka, T., Vajteršic, M.: Parallel rare term vector replacement: fast and effective dimensionality reduction for text. J. Parallel Distrib. Comput. 73, 341–351 (2013)

    Article  MATH  Google Scholar 

  9. Maaten, L.V.D., Postma, E.: Dimensionality Reduction: A Comparative Review. TiCC, Tilburg University, Tilburg (2009)

    Google Scholar 

  10. Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley, New York (2000)

    MATH  Google Scholar 

  11. Tenenbaum, J.B., Silva, Vd, Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)

    Article  Google Scholar 

  12. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensional reduction and data representation. Neural Comput. 15, 1373–1396 (2000)

    Article  MATH  Google Scholar 

  13. Axler, S., Ribet, K.A.: An Introduction to Manifolds, 2nd edn. Universitext: Springer, Berlin (2008)

    Google Scholar 

  14. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)

    Article  Google Scholar 

  15. Aeini, F., Eftekhari-Moghadam, A.M., Mahmoudi, F.: Non linear dimensional reduction method based on supervised neighborhood graph. In: 7th International Symposium on Telecommunications (IST’2014), IEEE, pp. 35–40 (2014)

  16. Cheng, J., Liu, Q., Lu, H., Chen, Y.W.: Supervised kernel locality preserving projections for face recognition. Neurocomputing 67, 443–449 (2005)

    Article  Google Scholar 

  17. Geng, X., Zhan, D.C., Zhou, Z.H.: Supervised nonlinear dimensionality reduction for visualization and classification. IEEE Trans. Syst. Man Cybern. Part B Cybern. 35(6), 1098–1107 (2005)

    Article  Google Scholar 

  18. Raducanu, B., Dornaika, F.: A supervised non-linear dimensionality reduction approach for manifold learning. Pattern Recognit. 45, 2432–2444 (2012)

    Article  MATH  Google Scholar 

  19. Jolliffe, I.T.: Principal Component Analysis, 2nd edn. Springer, New York (2002)

    MATH  Google Scholar 

  20. Fukunnaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, San Diego (1990)

    Google Scholar 

  21. Cox, T.F., Hart, P.E.: Multidimensional Scaling. Chapman and Hall, London (1994)

    MATH  Google Scholar 

  22. Silva, D., Tenenbaum, J.B.: Global versus local methods in nonlinear dimensionality reduction, In: Advances in Nerual Information Processing Systems (NIPS), pp. 705–712 (2002)

  23. Weinberger, K., Saul, L.K.: Unsupervised learning of image manifolds by semidefinite programming. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 988–995 (2004)

  24. Saul, L.K., Roweis, S.T.: Think globally, fit locally: unsupervised learning of low dimensional manifold. J. Mach. Learn. Res. 4, 119–155 (2003)

    MathSciNet  MATH  Google Scholar 

  25. Wen, G., Jiang, L., Shadbolt, N.R.: Using graph algebra to optimize neighborhood for isometric mapping. In: IJCAI, pp. 2398–2403 (2007)

  26. Vlachos, M., Domeniconi, M., Gunopulos, D., Kollios, G., Koudas, N.: Non-linear dimensionality reduction techniques for classification and visualization. In: Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Edmonton, AB, Canada, pp. 645–651 (2002)

  27. Ridder, D.D., Kouropteva, O., Okun, O., Pietikainen, M., Duin, R.P.W.: Supervised locally linear embedding. In: Artificial Neural Networks and Neural Information Processing-ICANN/ICONIP 2003, pp. 333–341, Springer (2003)

  28. Xian-Fa, C., Gui-Hua, W., Jia, W., Jie, L.: Enhanced supervised locality preserving projections for face recognition. In: International Conference on Machine Learning and Cybernetics, IEEE (2011). https://doi.org/10.1109/ICMLC.2011.6017017

  29. Ptucha, R., Savakis, A.: Manifold based sparse representation for facial understanding in natural images. Image Vision Comput. 31, 365–378 (2013)

    Article  Google Scholar 

  30. Schölkopf, B., Smola, A., Müller, K.-R.: Kernel principal component analysis. In: Artificial Neural Networks-ICANN’97, pp. 583–588. Springer, Lausanne (1997)

  31. Hettiarachchi, R., Peters, J.F.: Multi-manifold LLE learning in pattern recognition. Pattern Recognit. 48(9), 2947–2960 (2015)

    Article  Google Scholar 

  32. Omranpour, H., Ghidary, S.S.: Manifold based map representation for mobile robot using Euclidean data difference dimension reduction. Eng. Appl. Artif. Intell. 45, 234–245 (2015)

    Article  Google Scholar 

  33. Lee, C.-S., Elgammal, A., Torki, M.: Learning representations from multiple manifolds. Pattern Recognit. 50, 74–87 (2016)

    Article  Google Scholar 

  34. Nane, S.A., Nayar, S.K., Murase, H.: Columbia object image library (coil-20). Technical Report CUCS-005-96 (1996)

  35. Gao, W., et al.: The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 38(1), 149–161 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer and Information Technology Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran

    Faraein Aeini & Amir Masoud Eftekhari Moghadam

  2. Data Scientist Advanced Analytics Department, General Motors, Warren, MI, USA

    Fariborz Mahmoudi

Authors
  1. Faraein Aeini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir Masoud Eftekhari Moghadam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fariborz Mahmoudi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amir Masoud Eftekhari Moghadam.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aeini, F., Eftekhari Moghadam, A.M. & Mahmoudi, F. Supervised hierarchical neighborhood graph construction for manifold learning. SIViP 12, 799–807 (2018). https://doi.org/10.1007/s11760-017-1222-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-017-1222-4

Keywords

Search

Navigation