Abstract
Graph-based semi-supervised learning’s inherent ability to exploit the underlying structure of data distribution for supplementing label propagation has gained momentum over recent years. However, its effectiveness highly relies on the graph's structure quality and deteriorates when dealing with high dimensional, noisy, unevenly distributed data thus, necessitating adaptivity with sparsity in graph construction. To achieve this, an Exponentially Regularized Optimal Sparse Graph (EROSG) is introduced that inculcates these characteristics by exploring local connectivity ensuring efficient label propagation with reduced complexity. Accordingly, EROSG constructs the affinity matrix using a novel distance metric to widen the sample-wise interclass deviation and strengthen the local connectivity. The resulting affinity matrix is then optimized by Lagrangian multipliers with non-negative and SoftMax constraints to yield the adaptive sparse graph facilitating label propagation. Extensive analysis of EROSG on diverse datasets demonstrates consistent and superior accuracy of over 93% with a minimum availability of 5–10% of labeled data which is lacking in its competitors. Also, EROSG’s parameter-free nature lessens realization complexity emphasizing the need of the hour.
Graphical abstract
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
Publicly available datasets are utilized for performance analysis of the presented methodology.
References
Berton L, Lopes ADA (2014) Graph construction based on labeled instances for semi-supervised learning. Proceedings - International Conference on Pattern Recognition 2477–2482. https://doi.org/10.1109/ICPR.2014.428
Chong Y, Ding Y, Yan Q, Pan S (2020) Graph-based semi-supervised learning: a review. Neurocomputing 408:216–230. https://doi.org/10.1016/J.NEUCOM.2019.12.130
Deng J, Yu JG (2021) A simple graph-based semi-supervised learning approach for imbalanced classification. Pattern Recognit 118:108026. https://doi.org/10.1016/j.patcog.2021.108026
Nie F, Shi S, Li X (2020) Semi-supervised learning with auto-weighting feature and adaptive graph. IEEE Trans Knowl Data Eng 32:1167–1178. https://doi.org/10.1109/TKDE.2019.2901853
Chen C, Wang Z, Wu J et al (2021) Interactive graph construction for graph-based semi-supervised learning. IEEE Trans Vis Comput Graph 27:3701–3716. https://doi.org/10.1109/TVCG.2021.3084694
Song Z, Yang X, Xu Z, King I (2023) Graph-based semi-supervised learning: a comprehensive review. IEEE Trans Neural Netw Learn Syst 34(11):8174–8194. https://doi.org/10.1109/TNNLS.2022.3145691
Huang J, Nie F, Huang H (2015) A new simplex sparse learning model to measure data similarity for clustering. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. pp. 3569–3575
Hua Z, Yang Y (2022) Robust and sparse label propagation for graph-based semi-supervised classification. Appl Intell 52:3337–3351. https://doi.org/10.1007/S10489-021-02360-Z
Nie F, Xiang S, Liu Y, Zhang C (2010) A general graph-based semi-supervised learning with novel class discovery. Neural Comput Appl 19:549–555. https://doi.org/10.1007/s00521-009-0305-8
Miquilini P, Rossi RG, Quiles MG et al (2017) Automatically design distance functions for graph-based semi-supervised learning. Proceedings - 16th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, 11th IEEE International Conference on Big Data Science and Engineering and 14th IEEE International Conference on Embedded Software and Systems 933–940. https://doi.org/10.1109/Trustcom/BigDataSE/ICESS.2017.333
Chen Z, Cao H, Chang KCC (2020) GraphEBM: Energy-based graph construction for semi-supervised learning. In: Proceedings - IEEE International Conference on Data Mining, ICDM. Institute of Electrical and Electronics Engineers Inc., pp 62–71. https://doi.org/10.1109/ICDM50108.2020.00015
Ma J, Xiao B, Deng C (2020) Graph based semi-supervised classification with probabilistic nearest neighbors. Pattern Recognit Lett 133:94–101. https://doi.org/10.1016/j.patrec.2020.01.021
Nie F, Dong X, Li X (2021) Unsupervised and semisupervised projection with graph optimization. IEEE Trans Neural Netw Learn Syst 32:1547–1559. https://doi.org/10.1109/TNNLS.2020.2984958
Li S, Fu Y (2015) Learning balanced and unbalanced graphs via low-rank coding. IEEE Trans Knowl Data Eng 27:1274–1287. https://doi.org/10.1109/TKDE.2014.2365793
Peng Y, Lu BL, Wang S (2015) Enhanced low-rank representation via sparse manifold adaption for semi-supervised learning. Neural Netw 65:1–17. https://doi.org/10.1016/j.neunet.2015.01.001
Nie F, Wang X, Jordan MI, Huang H (2016) The constrained laplacian rank algorithm for graph-based clustering. 30th AAAI Conference on Artificial Intelligence, AAAI 2016 1969–1976. https://doi.org/10.1609/aaai.v30i1.10302
Kang Z, Peng C, Cheng Q et al (2021) Structured graph learning for clustering and semi-supervised classification. Pattern Recognit 110:107627. https://doi.org/10.1016/j.patcog.2020.107627
Nie F, Xu D, Tsang IWH, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19:1921–1932. https://doi.org/10.1109/TIP.2010.2044958
Iosifidis A, Tefas A, Pitas I (2014) Regularized extreme learning machine for multi-view semi-supervised action recognition. Neurocomputing 145:250–262. https://doi.org/10.1016/j.neucom.2014.05.036
Wang W, Yan Y, Nie F et al (2018) Flexible manifold learning with optimal graph for image and video representation. IEEE Trans Image Process 27:2664–2675. https://doi.org/10.1109/TIP.2018.2810515
Pang Y, Zhou B, Nie F (2019) Simultaneously learning neighborship and projection matrix for supervised dimensionality reduction. IEEE Trans Neural Netw Learn Syst 30:2779–2793. https://doi.org/10.1109/TNNLS.2018.2886317
Zhang Q, Chu T, Zhang C (2021) Semi-supervised graph based embedding with non-convex sparse coding techniques. IEEE Trans Knowl Data Eng 33:2193–2207. https://doi.org/10.1109/TKDE.2019.2953668
Yan S, Wang H (2009) Semi-supervised learning by sparse representation. Society for Industrial and Applied Mathematics - 9th SIAM International Conference on Data Mining 2009, Proceedings in Applied Mathematics 2:788–797. https://doi.org/10.1137/1.9781611972795.68
Fang X, Xu Y, Li X et al (2015) Learning a nonnegative sparse graph for linear regression. IEEE Trans Image Process 24:2760–2771. https://doi.org/10.1109/TIP.2015.2425545
Madhu C, Sudhakar MS (2024) Dialectic feature-based fuzzy graph learning for label propagation assisting text classification. IEEE Transactions on Fuzzy Systems. https://doi.org/10.1109/TFUZZ.2024.3421595
Madhu C, Sudhakar MS (2023) An interpretable fuzzy graph learning for label propagation assisting data classification. 1–14. https://doi.org/10.1109/TFUZZ.2023.3323093
Li M, Zhang X, Wang X (2010) An improved learning with local and global consistency. 2010 Chinese Control and Decision Conference, CCDC 2010 1152–1156. https://doi.org/10.1109/CCDC.2010.5498148
Wang Z, Nie F, Wang R et al (2021) Local structured feature learning with dynamic maximum entropy graph. Pattern Recognit 111:107673. https://doi.org/10.1016/j.patcog.2020.107673
Hamilton WL (2020) Graph representation learning. Morgan \& Claypool Publishers, Cham
Dokmanic I, Parhizkar R, Ranieri J, Vetterli M (2015) Euclidean distance matrices: essential theory, algorithms, and applications. IEEE Signal Process Mag 32:12–30
Zubaedah R, Xaverius F, Jayawardana H, Hidayat SH (2020) Comparing euclidean distance and nearest neighbor algorithm in an expert system for diagnosis of diabetes mellitus. Enferm Clin 30:374–377
Fang X, Xu Y, Li X et al (2015) Learning a nonnegative sparse graph for linear regression. IEEE Trans Image Process 24:2760–2771
Kanimozhi M, Sudhakar MS (2023) A local-global shape characterization scheme using quadratic Bezier triangle aiding retrieval. Digit Signal Process: Rev J 133:103893. https://doi.org/10.1016/j.dsp.2022.103893
Wang Y, Yao H, Zhao S (2016) Auto-encoder based dimensionality reduction. Neurocomputing 184:232–242
Ramamurthy M, Robinson YH, Vimal S, Suresh A (2020) Auto encoder based dimensionality reduction and classification using convolutional neural networks for hyperspectral images. Microprocess Microsyst 79:103280
Liu Y, Li Y, Tan X et al (2021) Local discriminant preservation projection embedded ensemble learning based dimensionality reduction of speech data of Parkinson’s disease. Biomed Signal Process Control 63:102165
Guo T, Yu K, Aloqaily M, Wan S (2022) Constructing a prior-dependent graph for data clustering and dimension reduction in the edge of AIoT. Futur Gener Comput Syst 128:381–394
Yadav RK, Abhishek VS, Venkatesan S (2021) Cross-covariance based affinity for graphs. Appl Intell 51:3844–3864
Singh D, Singh B (2020) Investigating the impact of data normalization on classification performance. Appl Soft Comput 97:105524. https://doi.org/10.1016/j.asoc.2019.105524
Aksoy S, Haralick RM (2001) Feature normalization and likelihood-based similarity measures for image retrieval. Pattern Recognit Lett 22:563–582. https://doi.org/10.1016/S0167-8655(00)00112-4
Campos GO, Zimek A, Sander J, Campello RJ, Micenková B, Schubert E, Assent I, Houle ME (2016) On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study. Data Min Knowl Disc 30:891–927. https://doi.org/10.1007/s10618-015-0444-8
Wang F, Zhu L, Xie L, Zhang Z, Zhong M (2021) Label propagation with structured graph learning for semi-supervised dimension reduction. Knowledge-Based Syst 225:107130. https://doi.org/10.1016/j.knosys.2021.107130
Luo Y, Wong Y, Kankanhalli M, Zhao Q (2019) G-softmax: improving intraclass compactness and interclass separability of features. IEEE Trans Neural Netw Learn Syst 31:685–699
Li Q (2023) A comprehensive survey of sparse regularization: fundamental, state-of-the-art methodologies and applications on fault diagnosis. Expert Syst Appl 229:120517
Lyons M, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with Gabor wavelets. Proceedings - 3rd IEEE International Conference on Automatic Face and Gesture Recognition, FG 1998 200–205. https://doi.org/10.1109/AFGR.1998.670949
Lyons MJ (1999) Automatic classification of single facial images. IEEE Trans Pattern Anal Mach Intell 21:1357–1362. https://doi.org/10.1109/34.817413
Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23:643–660. https://doi.org/10.1109/34.927464
Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16:550–554. https://doi.org/10.1109/34.291440
Nene S, Nayar S, Murase H (1996) Columbia object image library (COIL-20). Tech Rep 95:223–303
Ali N, Zafar B, Riaz F et al (2018) A hybrid geometric spatial image representation for scene classification. PLoS One 13:e0203339
Gui J, Hu R, Zhao Z, Jia W (2014) Semi-supervised learning with local and global consistency. Int J Comput Math 91:2389–2402. https://doi.org/10.1080/00207160.2013.831082
Nie F, Wang H, Huang H, Ding C (2011) Unsupervised and semi-supervised learning via ℓ 1-norm graph. Proceedings of the IEEE International Conference on Computer Vision 2268–2273. https://doi.org/10.1109/ICCV.2011.6126506
Zhu X, Ghahramani Z, Lafferty JD (2003) Semi-supervised learning using Gaussian fields and harmonic functions. In Proceedings of the 20th International Conference on Machine Learning (ICML-03) (pp. 912–919)
Li CG, Lin Z, Zhang H, Guo J (2015) Learning semi-supervised representation towards a unified optimization framework for semi-supervised learning. Proceedings of the IEEE International Conference on Computer Vision 2015 Inter:2767–2775. https://doi.org/10.1109/ICCV.2015.317
Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. Proc Twentieth Int Conf Mach Learn 2:912–919
Nie F, Cai G, Li X (2017) Multi-view clustering and semi-supervised classification with adaptive neighbours. 31st AAAI Conference on Artificial Intelligence, AAAI 2017 2408–2414. https://doi.org/10.1609/aaai.v31i1.10909
Nie F, Dong X, Li X (2020) Unsupervised and semisupervised projection with graph optimization. IEEE Trans Neural Netw Learn Syst 32:1547–1559
Wang F, Zhu L, Xie L et al (2021) Label propagation with structured graph learning for semi-supervised dimension reduction. Knowl Based Syst 225:107130
Li Y, Bai L (2023) Label propagation based on bipartite graph. Neural Process Lett 0–18. https://doi.org/10.1007/s11063-023-11282-5
Hua Z, Yang Y (2022) Robust and sparse label propagation for graph-based semi-supervised classification. Applied Intelligence. https://doi.org/10.1007/s10489-022-03666-y
Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. Proceedings of the IEEE 11th International Conference on Computer Vision (ICCV’07) October 2007 Rio de Janeiro. Brazil IEEE 1–7. https://doi.org/10.1109/iccv.2007.4408856
Ren S, Gu X, Yuan P, Xu H (2018) An iterative paradigm of joint feature extraction and labeling for semi-supervised discriminant analysis. Neurocomputing 273:466–480
Nie F, Xu D, Tsang IW-H, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19:1921–1932
Zhu R, Dornaika F, Ruichek Y (2020) Semi-supervised elastic manifold embedding with deep learning architecture. Pattern Recognit 107:107425
Dornaika F, Hoang VT (2023) Deep data representation with feature propagation for semi-supervised learning. Int J Mach Learn Cybern 14:1303–1316
Zhu R, Dornaika F, Ruichek Y (2019) Learning a discriminant graph-based embedding with feature selection for image categorization. Neural Netw 111:35–46
Huang Y, Xu D, Nie F (2012) Semi-supervised dimension reduction using trace ratio criterion. IEEE Trans Neural Netw Learn Syst 23:519–526. https://doi.org/10.1109/TNNLS.2011.2178037
Wang D, Nie F, Huang H (2014) Large-scale adaptive semi-supervised learning via unified inductive and transductive model. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 482–491. https://doi.org/10.1145/2623330.2623731
Zhou Y, Sun S (2015) Semisupervised tangent space discriminant analysis. Math Probl Eng 2015. https://doi.org/10.1155/2015/706180
Liu H, Han J, Nie F (2017) Semi-supervised orthogonal graph embedding with recursive projections. IJCAI Int Joint Conf Artif Intell 0:2308–2314. https://doi.org/10.24963/ijcai.2017/321
Nie F, Wang Z, Wang R, Li X (2021) Adaptive local embedding learning for semi-supervised dimensionality reduction. IEEE Trans Knowl Data Eng 34:4609–4621. https://doi.org/10.1109/TKDE.2021.3049371
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
M, K., MS, S. SparseGraphX: exponentially regularized optimal sparse graph for enhanced label propagation. Appl Intell 55, 131 (2025). https://doi.org/10.1007/s10489-024-06007-7
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06007-7