Skip to main content
SpringerLink
Log in

A regularized approach for supervised multi-view multi-manifold learning from unlabeled data

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, we combined two steps in a new multi-view multi-manifold learning algorithm that are essential for recognition tasks in the absence of class label information; first, we emphasize the first step of graph-based multi-view multi-manifold learning methods, i.e., select class-consistent neighbors from all available views. In multi-manifold space, the ideal neighborhood set is unidentified, and selection of a proper neighborhood set is not an easy task, especially if manifolds have some intersections. We describe each class of objects with continuous varying of pose angle as a relatively independent object-manifold. To find the object-manifolds, we utilize the transitivity of the similarity in the objects and use the TV-regularization to describe each object in a weighted sum of its class-consistent neighbors under different views. The proposed method aims to make a distinction between some objects with the same class and some objects with different classes that have similar views. The proposed method can be efficiently solved by an ADMM method. Second, we propose a regularized approach for supervised dimension reduction via discovering the discriminating information hidden in the data structure. Neighborhood selection and recognition accuracy experiments on COIL-20, CAS-PEAL, FEI, and ORL multi-view datasets have shown the excellent performance of our novel approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Suliman A, Omarov BS (2018) Applying Bayesian Regularization for Acceleration of Levenberg-Marquardt based Neural Network Training. International Journal of Interactive Multimedia and Artificial Intelligence 5(1):68–72

    Article  Google Scholar 

  2. Magdin M, Prikler F (2018) Real-Time Facial Expression Recognition Using Webcam and SDK Affectiva. International Journal of Interactive Multimedia and Artificial Intelligence 5(1):7–15

    Article  Google Scholar 

  3. Zhang Y, Ye D, Liu Y (2018) Robust locally linear embedding algorithm for machinery fault diagnosis. Neurocomputing 273(17):323–332

    Article  Google Scholar 

  4. Ren S et al (2018) An iterative paradigm of joint feature extraction and labeling for semi-supervised discriminant analysis. Neurocomputing 273(17):466–480

    Article  Google Scholar 

  5. Hu MW, Sun Z, Zhao S (2018) Kernel collaboration representation-based manifold regularized model for unconstrained face recognition. SIViP 12(5):925–932

    Article  Google Scholar 

  6. Yang M et al (2017) Joint regularized nearest points for image set based face recognition. Image Vis Comput 58:47–60

    Article  Google Scholar 

  7. Zhang Z, Mao J (2016) Jointly sparse neighborhood graph for multi-view manifold clustering. Neurocomputing 216(5):28–38

    Article  Google Scholar 

  8. Lai Z, Wan M, Jin Z (2011) Locality preserving embedding for face and handwriting digital recognition. Neural Comput & Applic 20:565–573

    Article  Google Scholar 

  9. Yan H et al (2014) Multi-feature multi-manifold learning for single-sample face recognition. Neurocomputing 143(2):134–143

    Article  Google Scholar 

  10. Chen W-J et al (2014) Manifold proximal support vector machine for semi-supervised classification. Appl Intell 40(4):623–638

    Article  Google Scholar 

  11. Belkin M, Niyogi P (2000) Laplacian eigenmaps for dimensional reduction and data representation. Neural Comput 15:1373–1396

    Article  MATH  Google Scholar 

  12. Zhang Z, Zha H (2002) Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment. SIAM Journal of Scientific Computing 26(1):313–338

    Article  MATH  Google Scholar 

  13. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323–2326

    Article  Google Scholar 

  14. Tenenbaum J, Silva V, Langford J (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Article  Google Scholar 

  15. Vlachos M, et al (2002) Non-linear dimensionality reduction techniques for classification and visualization. In: Proceedings of ACM Int. Conf. Knowl. Discovery Data Mining. ACM New York pp. 645–651

  16. Belkin M, Niyogi P (2002) Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Neural Information Processing Systems, pp. 585–591

  17. Hettiarachchi R, Peters JF (2015) Multi-manifold LLE learning in pattern recognition. Pattern Recogn 48(9):2947–2960

    Article  MATH  Google Scholar 

  18. Lee C-S, Elgammal A, Torki M (2016) Learning representations from multiple manifolds. Pattern Recogn 50:74–87

    Article  Google Scholar 

  19. Fan M et al (2016) Efficient isometric multi-manifold learning based on the self-organizing method. Inf Sci 345:325–339

    Article  Google Scholar 

  20. Yang B, Xiang M, Zhang Y (2016) Multi-manifold discriminant Isomap for visualization and classification. Pattern Recogn 55:215–230

    Article  MATH  Google Scholar 

  21. Li B, Li J, Zhang X-P (2015) Nonparametric discriminant multi-manifold learning for dimensionality reduction. Neurocomputing 152(25):121–126

    Article  Google Scholar 

  22. Li J et al (2016) Multi-manifold Sparse Graph Embedding for Multi-modal Image Classification. Neurocomputing 173(3):501–510

    Article  Google Scholar 

  23. Sun S (2013) A survey of multi-view machine learning. Neural Comput & Applic 23:2031–2038

    Article  Google Scholar 

  24. Li Y et al (2016) Manifold regularized multi-view feature selection for social image annotation. Neurocomputing 204(5):135–141

    Article  Google Scholar 

  25. Nane SA, Nayar SK, Murase H (1996) Columbia object image library (COIL-20). Department of Computer Science: Columbia University

  26. Gao W, Cao B, Shan S (2008) The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans 38(1)

  27. Gao W et al (2008) The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems And Humans 38(1):149–161

    Article  Google Scholar 

  28. Wang L, Zhang Y, Feng J (2005) On the Euclidean distance of images. IEEE Trans Pattern Anal Mach Intell 27(8):1334–1339

    Article  Google Scholar 

  29. Geng X, Zhan DC, Zhou ZH (2005) Supervised Nonlinear Dimensionality Reduction for Visualization and Classification. IEEE Transactions On Systems, Man, And Cybernetics—Part B: Cybernetics 35(6):1098–1107

    Article  Google Scholar 

  30. Raducanu B, Dornaika F (2012) A supervised non-linear dimensionality reduction approach for manifold learning. Pattern Recogn 45:2432–2444

    Article  MATH  Google Scholar 

  31. Aeini F, Eftekhari Moghadam AM, Mahmoudi F (2014) Non linear dimensional reduction method based on supervised neighborhood graph. In: 7th International Symposium on Telecommunications (IST'2014). IEEE: Tehran. p. 35–40

  32. Ridder DD, et al (2003) Supervised locally linear embedding. In: Artificial Neural Networks and Neural Information Processing-ICANN/ICONIP 2003. 2003, Springer. p. 333–341

  33. Zhang Z, Chow TWS, Zhao M (2013) M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction. IEEE Transactions on Cybernetics 43(1):180–191

    Article  Google Scholar 

  34. Aeini F, Eftekhari Moghadam AM, Mahmoud F (2018) Supervised hierarchical neighborhood graph construction for manifold learning. SIViP 12(4):799–807

    Article  Google Scholar 

  35. He X, Niyogi P (2004) Locality preserving projections. In: NIPS'03 Proceedings of the 16th International Conference on Neural Information Processing Systems. Whistler, British Columbia, Canada p. 153–160

  36. Kokiopoulou E, Saad Y (2007) Orthogonal neighborhood preserving projections: A projection-based dimensionality reduction technique. IEEE Trans Pattern Anal Mach Intell 29(12):2143–2156

    Article  Google Scholar 

  37. Cheng J et al (2005) Supervised kernel locality-preserving projections for face recognition. Neurocomputing 67:443–449

    Article  Google Scholar 

  38. Fa X, et al (2011) Enhanced supervised locality preserving projections for face recognition, in International Conference on Machine Learning and Cybernetics

  39. Fan M, et al (2012) Isometric multi-manifold learning for feature extraction. In: Proceedings of the Twelfth IEEE International Conference on Data Mining (ICDM). p. 241–250

  40. Lu J, Tan Y-P, Wang G (2013) Discriminative Multimanifold Analysis for Face Recognition from a Single Training Sample per Person. IEEE Trans Pattern Anal Mach Intell 35(1):39–51

    Article  Google Scholar 

  41. Liu J, Li B, Zhang W-S (2012) Feature extraction using maximum variance sparse mapping. Neural Comput & Applic 21:1827–1833

    Article  Google Scholar 

  42. Feng P, Bresler Y (1996) Spectrum-blind minimum-rate sampling and reconstruction of multi-band signals. ICASSP3:1688–1691

  43. Obozinski G, Taskar B, Jordan M (2010) Support union recovery in high-dimensional multi variate regression. Stat Comput 20(2):231–252

    Article  MathSciNet  Google Scholar 

  44. Fornasier M, Pitolli F (2008) Adaptive iterative thresholding algorithms for magnetoence phalography (MEG). Comput Appl Math 211:386–395

    Article  MATH  Google Scholar 

  45. Nie F et al (2010) Efficient and robust feature selection via joint ℓ2,1-norms minimization. Adv Neural Inf Proces Syst 2:1813–1821

    Google Scholar 

  46. Maaten LJPVD, Postma EO, Herik HJVD (2009) Dimensionality Reduction: A Comparative Review. Mach Learn Res 10(1–41):66–71

    Google Scholar 

  47. Aeini F, Eftekhari Moghadam AM, Mahmoudi F (2018) A regularized approach for unsupervisedmulti-viewmulti-manifold learning. Signal, Image and Video Processing 1–9

  48. Xu Y et al (2010) LPP solution schemes for use with face recognition. Pattern Recogn 43:4165–4176

    Article  MATH  Google Scholar 

  49. Evgeniou T et al (2002) Regularization and statistical learning theory for data analysis. Computational Statistics & Data Analysis 38(4):421–432

    Article  MathSciNet  MATH  Google Scholar 

  50. Belkin M, Niyogi P, Sindhwani V (2006) Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples. J Mach Learn Res 7:2399–2434

    MathSciNet  MATH  Google Scholar 

  51. Schölkopf B, Herbrich R, Smola AJ (2001) A Generalized Representer Theorem. COLT 2001: Computational Learning Theory, p. 416–426

  52. Thomaz CE, Giraldi GA (2010) A new ranking method for Principal Components Analysis and its application to face image analysis. Image Vis Comput 28(6):902–913

    Article  Google Scholar 

  53. AT&T Laboratories Cambridge (2002) The ORL database of faces. "http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html. (Online;accessed 23.12.2014). Cambridge University Computer Laboratory

  54. Tran L et al (2015) Adaptive graph construction for Isomap manifold learning. Article (PDF Available). Proceedings of SPIE - The International Society for Optical Engineering 1:1–7

    Google Scholar 

  55. Örnek C, Vural E (2019) Nonlinear supervised dimensionality reduction via smooth regular embeddings. Pattern Recogn 87:55–66

    Article  Google Scholar 

  56. Yan Y et al (2018) Face recognition algorithm using extended vector quantization histogram features. PLoS One 13(1)

  57. Zhang Z, Song G, Wu J (2014) A Novel Two-Stage Illumination Estimation Framework for Expression Recognition. Sci World J 2014:1–12

    Google Scholar 

  58. Tsai Y-H et al (2018) Robust in-plane and out-of-plane face detection algorithm using frontal face detector and symmetry extension. Image Vis Comput 78:26–41

    Article  Google Scholar 

  59. Abhishree TM et al (2015) Face Recognition Using Gabor Filter Based Feature Extraction with Anisotropic Diffusion as a Pre-processing Technique. Procedia Computer Science 45:312–321

    Article  Google Scholar 

  60. Raducanu B, Dornaika F (2014) Embedding new observations via sparse-coding for non-linear manifold learning. Pattern Recogn 47(1)

  61. Samaria F, Harter A (1994) Parameterisation of a Stochastic Model for Human Face Identification, in Proceedings of 2nd IEEE Workshop on Applications of Computer Vision, Sarasota FL. IEEE: Sarasota

  62. Wasserman PD (1993) Advanced methods in nerual computing. Van Nostrand reinhold, New York

    Google Scholar 

  63. Nane, S.A., S.K. Nayar, and H. Murase (1996) Columbia object image library (coil-20). Technical Report CUCS-005-96

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Sari Branch, Islamic Azad University, Sari, Iran

    Faraein Aeini

  2. Faculty of Computer and Information Technology Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran

    Amir Masoud Eftekhari Moghadam

  3. Data Scientist Advanced Analytics Department, General Motors, Warren, MI, USA

    Fariborz Mahmoudi

Authors
  1. Faraein Aeini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir Masoud Eftekhari Moghadam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fariborz Mahmoudi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amir Masoud Eftekhari Moghadam.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aeini, F., Eftekhari Moghadam, A.M. & Mahmoudi, F. A regularized approach for supervised multi-view multi-manifold learning from unlabeled data. Appl Intell 49, 3173–3187 (2019). https://doi.org/10.1007/s10489-019-01411-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01411-w

Keywords

Search

Navigation