Skip to main content
SpringerLink
Log in

Joint graph and reduced flexible manifold embedding for scalable semi-supervised learning

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Recently, graph-based semi-supervised learning (GSSL) has received much attention. On the other hand, less attention has been paid to the problem of large-scale GSSL for inductive multi-class classification. Existing scalable GSSL methods rely on a hard linear constraint. They cannot predict the labelling of test samples, or use predefined graphs, which limits their applications and performance. In this paper, we propose an inductive algorithm that can handle large databases by using anchors. The main contribution compared to existing scalable semi-supervised models is the integration of the anchor graph computation into the learned model. We develop a criterion to jointly estimate the unlabeled sample labels, the mapping of the feature space to the label space, and the affinity matrix of the anchor graph. Furthermore, the fusion of labels and features of anchors is used to construct the graph. Using the projection matrix, it can also predict the labels of the test samples by linear transformation. Experimental results on the large datasets NORB, RCV1 and Covtype show the effectiveness, scalability and superiority of the proposed method. The code of the proposed method can be found at the following link https://github.com/ZoulfikarIB/SGRFME .

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

The data that support the findings of this study are available upon reasonable request.

Notes

  1. https://cs.nyu.edu/~ylclab/data/norb-v1.0-small/.

  2. http://archive.ics.uci.edu/ml/datasets/Covertype.

References

  • An L, Chen X, Yang S (2017) Multi-graph feature level fusion for person re-identification. Neurocomputing 259:39–45

    Article  Google Scholar 

  • An J, Zhao X, Shi M, Liu X, Guo J (2021) Joint neighborhood preserving and projected clustering for feature extraction. Neurocomputing 488:572–580

    Article  Google Scholar 

  • Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(85):2399–2434

    MathSciNet  MATH  Google Scholar 

  • Chen H-T, Chang H-W, Liu T-L (2005) Local discriminant embedding and its variants. In: 2005 IEEE Computer Society conference on computer vision and pattern recognition (CVPR’05), 2005, vol 2, pp 846–853

  • Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7(08):1687–1712

    MathSciNet  MATH  Google Scholar 

  • Cui B, Xie X, Hao S, Cui J, Lu Y (2018) Semi-supervised classification of hyperspectral images based on extended label propagation and rolling guidance filtering. Remote Sens 10(4):515

    Article  Google Scholar 

  • de Sousa CAR (2016) An inductive semi-supervised learning approach for the local and global consistency algorithm. In: 2016 International joint conference on neural networks (IJCNN), 2016, pp 4017–4024

  • Deng J, Yu J-G (2021) A simple graph-based semi-supervised learning approach for imbalanced classification. Pattern Recognit 118:108026

    Article  Google Scholar 

  • Dornaika F, Traboulsi YE, Assoum A (2016) Inductive and flexible feature extraction for semi-supervised pattern categorization. Pattern Recognit 60:275–285

    Article  MATH  Google Scholar 

  • Dornaika F, Dahbi R, Bosaghzadeh A, Ruichek Y (2017) Efficient dynamic graph construction for inductive semi-supervised learning. Neural Netw 94:192–203

    Article  MATH  Google Scholar 

  • Dornaika F, Baradaaji A, El Traboulsi Y (2021) Joint label inference and discriminant embedding. IEEE Trans Neural Netw Learn Syst 33(9):4413–4423

    Article  MathSciNet  MATH  Google Scholar 

  • Dornaika F, Baradaaji A, El Traboulsi Y (2021) Semi-supervised classification via simultaneous label and discriminant embedding estimation. Inf Sci 546:146–165

    Article  MathSciNet  MATH  Google Scholar 

  • Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30. Curran Associates, Inc., Red Hook

    Google Scholar 

  • He Z, Xia K, Li T, Zu B, Yin Z, Zhang J (2021) A constrained graph-based semi-supervised algorithm combined with particle cooperation and competition for hyperspectral image classification. Remote Sens 13(2):193

    Article  Google Scholar 

  • Jia J, Ruan Q, Jin Y, An G, Ge S (2020) View-specific subspace learning and re-ranking for semi-supervised person re-identification. Pattern Recognit 108:107568

    Article  Google Scholar 

  • Jian M, Jung C (2021) Semi-supervised kernel matrix learning using adaptive constraint-based seed propagation. Pattern Recognit 112:107750

    Article  Google Scholar 

  • Kang Z, Peng C, Cheng Q, Liu X, Peng X, Xu Z, Tian L (2021) Structured graph learning for clustering and semi-supervised classification. Pattern Recognit 110:107627

    Article  Google Scholar 

  • Kang Z, Peng C, Cheng Q, Liu X, Peng X, Xu Z, Tian L (2021) Structured graph learning for clustering and semi-supervised classification. Pattern Recognit 110:107627

    Article  Google Scholar 

  • Kim K-H, Choi S (2014) Label propagation through minimax paths for scalable semi-supervised learning. Pattern Recognit Lett 45:17–25

    Article  Google Scholar 

  • Liu W, He J, Chang S-F (2010) Large graph construction for scalable semi-supervised learning. In: Proceedings of the 27th international conference on international conference on machine learning, ICML’10, 2010. Omnipress, Madison, pp 679–686

  • Liu Y, Shi H, Du H, Zhu R, Wang J, Zheng L, Mei T (2022) Boosting semi-supervised face recognition with noise robustness. IEEE Trans Circuits Syst Video Technol 32(2):778–787

    Article  Google Scholar 

  • Long Y, Li Y, Wei S, Zhang Q, Yang C (2019) Large-scale semi-supervised training in deep learning acoustic model for ASR. IEEE Access 7:133615–133627

    Article  Google Scholar 

  • Nie F, Xu D, Tsang IW, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932

    Article  MathSciNet  MATH  Google Scholar 

  • Nie F, Wang X, Jordan IM, Huang H (2016) The constrained Laplacian rank algorithm for graph-based clustering. In: AAAI conference on artificial intelligence, 2016

  • Nie F, Cai G, Li X (2017) Multi-view clustering and semi-supervised classification with adaptive neighbours. In: Thirty-first AAAI conference on artificial intelligence, 2017

  • Qiu S, Nie F, Xu X, Qing C, Xu D (2019) Accelerating flexible manifold embedding for scalable semi-supervised learning. IEEE Trans Circuits Syst Video Technol 29(9):2786–2795

    Article  Google Scholar 

  • Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  • Sindhwani V, Niyogi P (2005) Linear manifold regularization for large scale semi-supervised learning. In: Proceedings of the 22nd ICML workshop on learning with partially classified training data, 2005

  • Sindhwani V, Niyogi P, Belkin M, Keerthi S (2005) Linear manifold regularization for large scale semi-supervised learning. In: Proceedings of the 22nd ICML workshop on learning with partially classified training data, 2005, vol 1

  • Song Z, Yang X, Xu Z, King I (2021) Graph-based semi-supervised learning: a comprehensive review. CoRR, abs/2102.13303

  • Tu E, Wang Z, Yang J, Kasabov N (2022) Deep semi-supervised learning via dynamic anchor graph embedding in latent space. Neural Netw 146:350–360

    Article  Google Scholar 

  • Wang M, Fu W, Hao S, Tao D, Wu X (2016a) Scalable semi-supervised learning by efficient anchor graph regularization. IEEE Trans Knowl Data Eng 28(7):1864–1877

    Article  Google Scholar 

  • Wang M, Fu W, Hao S, Tao D, Wu X (2016b) Scalable semi-supervised learning by efficient anchor graph regularization. IEEE Trans Knowl Data Eng 28(7):1864–1877

    Article  Google Scholar 

  • Wang F, Zhu L, Xie L, Zhang Z, Zhong M (2021) Label propagation with structured graph learning for semi-supervised dimension reduction. Knowl Based Syst 225:107130

    Article  Google Scholar 

  • Wang Z, Zhang L, Wang R, Nie F, Li X (2022) Semi-supervised learning via bipartite graph construction with adaptive neighbors. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2022.3151315

    Article  Google Scholar 

  • Wu G, Li Y, Xi J, Yang X, Liu X (2012) Local learning integrating global structure for large scale semi-supervised classification. In: 2012 8th International conference on natural computation, 2012, pp 1044–1049

  • Wu X, Zhao L, Akoglu L (2019) A quest for structure: jointly learning the graph structure and semi-supervised classification

  • Yang S, Ienco D, Esposito R, Pensa RG (2021) ESA: a generic framework for semi-supervised inductive learning. Neurocomputing 447:102–117

    Article  Google Scholar 

  • Yi Y, Chen Y, Wang J, Lei G, Dai J, Zhang H (2020) Joint feature representation and classification via adaptive graph semi-supervised nonnegative matrix factorization. Signal Process Image Commun 89:115984

    Article  Google Scholar 

  • Yuan Y, Li X, Wang Q, Nie F (2021) A semi-supervised learning algorithm via adaptive Laplacian graph. Neurocomputing 426:162–173

    Article  Google Scholar 

  • Zaman SMK, Liang X (2021) An effective induction motor fault diagnosis approach using graph-based semi-supervised learning. IEEE Access 9:7471–7482

    Article  Google Scholar 

  • Zhan W, Zhang M-L (2017) Inductive semi-supervised multi-label learning with co-training. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’17, 2017. Association for Computing Machinery, New York, pp 1305–1314

  • Zhang Y-M, Huang K, Geng G-G, Liu C-L (2015) MTC: a fast and robust graph-based transductive learning method. IEEE Trans Neural Netw Learn Syst 26(9):1979–1991

    Article  MathSciNet  Google Scholar 

  • Zhang Z, Li F, Jia L, Qin J, Zhang L, Yan S (2018) Robust adaptive embedded label propagation with weight learning for inductive classification. IEEE Trans Neural Netw Learn Syst 29(8):3388–3403

    Article  MathSciNet  Google Scholar 

  • Zhang Z, Zhang Y, Liu G, Tang J, Yan S, Wang M (2019) Joint label prediction based semi-supervised adaptive concept factorization for robust data representation. IEEE Trans Knowl Data Eng 32(5):952–970

    Article  Google Scholar 

  • Zhao M, Zhang Y, Zhang Z, Liu J, Kong W (2019) ALG: adaptive low-rank graph regularization for scalable semi-supervised and unsupervised learning. Neurocomputing 370:16–27

    Article  Google Scholar 

  • Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. In: Thrun S, Saul LK, Schölkopf B (eds) Advances in neural information processing systems, vol 16. MIT Press, Cambridge, pp 321–328

    Google Scholar 

  • Zhu X, Lafferty J (2005) Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning, vol 1, pp 1052–1059

  • Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using Gaussian fields and harmonic functions. Int Conf Mach Learn 3(01):912–919

    Google Scholar 

  • Zhu R, Dornaika F, Ruichek Y (2021) Inductive semi-supervised learning with graph convolution based regression. Neurocomputing 434:315–322

    Article  Google Scholar 

  • Ziraki N, Dornaika F, Bosaghzadeh A (2022) Multiple-view flexible semi-supervised classification through consistent graph construction and label propagation. Neural Netw 146:174–180

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of the Basque Country UPV/EHU, San Sebastian, Spain

    Z. Ibrahim

  2. Shahid Rajaee Teacher Training University, Tehran, Iran

    A. Bosaghzadeh

  3. Ho Chi Minh City Open University, 97 Vo Van Tan, Ward Vo Thi Sau, District 3, Ho Chi Minh City, 70000, Vietnam

    F. Dornaika

Authors
  1. Z. Ibrahim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Bosaghzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. F. Dornaika
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Dornaika.

Ethics declarations

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Ethical approval

This article complies with ethical standards.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ibrahim, Z., Bosaghzadeh, A. & Dornaika, F. Joint graph and reduced flexible manifold embedding for scalable semi-supervised learning. Artif Intell Rev 56, 9471–9495 (2023). https://doi.org/10.1007/s10462-023-10397-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-023-10397-4

Keywords

Search

Navigation