Abstract
Graph-based embedding has attracted much attention in the fields of machine learning and pattern recognition. It is becoming an indispensable tool for data representation. It can be useful for all types of learning: unsupervised, semi-supervised, and supervised. In this correspondence, we present a graph-based, deep and flexible method for data representation with feature propagation. The presented framework ensures several desired features such as graph-based regularization, a flexible embedding model, graph-based feature propagation, and a deep learning architecture. The model can be learned layer by layer. In each layer, the nonlinear data representation and the unknown convolved data based regression are jointly estimated with a closed-form solution. We evaluate the proposed system on semi-supervised classification tasks using six public image datasets. These experiments demonstrate the effectiveness of the presented framework, which compares favorably to many competing semi-supervised approaches.



Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The data that support the findings of this study are available from the corresponding author, F. D., upon request.
References
Zhu R, Dornaika F, Ruichek Y (2019) Joint graph based embedding and feature weighting for image classification. Pattern Recognit 93:458–469
Zhu R, Dornaika F, Ruichek Y (2019) Learning a discriminant graph-based embedding with feature selection for image categorization. Neural Netw 111:35–46
Han C, Zhou D, Xie Y, Lei Y, Shi J (2021) Label propagation with multi-stage inference for visual domain adaptation. Knowl-Based Syst 216:106809
He F, Nie F, Wang R, Jia W, Zhang F, Li X (2020) Semisupervised band selection with graph optimization for hyperspectral image classification. IEEE Trans Geosci Remote Sensing 59:10298–10311
Zhao M, Zhang Y, Zhang Z, Liu J, Kong W (2019) Alg: adaptive low-rank graph regularization for scalable semi-supervised and unsupervised learning. Neurocomputing 370:16–27
Jia J, Ruan Q, Jin Y, An G, Ge S (2020) View-specific subspace learning and re-ranking for semi-supervised person re-identification. Pattern Recognit 108:107568
Yang Y, Zhan D-C, Wu Y-F, Liu Z-B, Xiong H, Jiang Y (2021) Semi-supervised multi-modal clustering and classification with incomplete modalities. IEEE Trans Knowl Data Eng 33(2):682–695
Zhu R, Dornaika F, Ruichek Y (2021) Inductive semi-supervised learning with graph convolution based regression. Neurocomputing 434:315–322
Fan D-P, Zhou T, Ji G-P, Zhou Y, Chen G, Fu H, Shen J, Shao L (2020) Inf-net: automatic COVID-19 lung infection segmentation from ct images. IEEE Trans Med Imaging 39(8):2626–2637
Hamid M (2019) Semi-supervised learning for plankton image classification. Master’s thesis, Lappeenranta-Lahti University of Technology LUT
Zhu R, Dornaika F, Ruichek Y (2020) Semi-supervised elastic manifold embedding with deep learning architecture. Pattern Recognit 107:107425
An J, Zhao X, Shi M, Liu X, Guo J (2021) Joint neighborhood preserving and projected clustering for feature extraction. Neurocomputing 488:572–580
Dornaika F, Baradaaji A, El Traboulsi Y (2021) Semi-supervised classification via simultaneous label and discriminant embedding estimation. Inform Sci 546:146–165
Hu Y, You H, Wang Z, Wang Z, Zhou E, Gao Y (2021) Graph-mlp: mode classification without message passing in graph
Jiang B, Zhang Z, Lin D, Tang J, Luo B (2019) Semi-supervised learning with graph learning-convolutional networks. In: IEEE/CVF conference on computer vision and pattern recognition, pp 11313–11320
Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. Adv Neural Inform Process Syst 16:321–328
Zhu X, Ghahramani Z, Lafferty JD (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: International conference on machine learning, pp 912–919
Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: IEEE International Conference on Computer Vision, pp 1–7
Chen H-T, Chang H-W, Liu T-L (2005) Local discriminant embedding and its variants. IEEE Conf Comput Vision Pattern Recognit 2:846–853
Hou C, Nie F, Li X, Yi D, Wu Y (2014) Joint embedding learning and sparse regression: a framework for unsupervised feature selection. IEEE Trans Cybernetics 44(6):793–804
Nie F, Cai G, Li J, Li X (2018) Auto-weighted multi-view learning for image clustering and semi-supervised classification. IEEE Trans Image Process 27(3):1501–1511
Nie F, Xu D, Tsang IW-H, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932
Dornaika F, El Traboulsi Y (2017) Margin based semi-supervised elastic embedding for face image analysis. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1313–1320
El Traboulsi Y, Dornaika F, Assoum A (2015) Kernel flexible manifold embedding for pattern classification. Neurocomputing 167:517–527
Nie F, Cai G, Li X (2017) Multi-view clustering and semi-supervised classification with adaptive neighbours. In: Thirty-First AAAI Conference on Artificial Intelligence
Dornaika F, Traboulsi YE (2019) Joint sparse graph and flexible embedding for graph-based semi-supervised learning. Neural Netw 114:91–95
Nie F, Wang Z, Wang R, Li X (2020) Submanifold-preserving discriminant analysis with an auto-optimized graph. IEEE Trans Cybernet 50(8):3682–3695
Nie F, Dong X, Li X (2020) Unsupervised and semisupervised projection with graph optimization. IEEE Trans Neural Netw Learn Syst 32:1547–1559
Nie F, Wang Z, Wang R, Li X (2021) Adaptive local embedding learning for semi-supervised dimensionality reduction. IEEE Trans Knowl Data Eng 34:4609–4621
Ziraki N, Dornaika F, Bosaghzadeh A (2022) Multiple-view flexible semi-supervised classification through consistent graph construction and label propagation. Neural Netw 146:174–180
Wang F, Zhu L, Xie L, Zhang Z, Zhong M (2021) Label propagation with structured graph learning for semi-supervised dimension reduction. Knowl-Based Syst 225:107130
Kang Z, Peng C, Cheng Q, Liu X, Peng X, Xu Z, Tian L (2021) Structured graph learning for clustering and semi-supervised classification. Pattern Recognit 110:107627
Yan W, Sun Q, Sun H, Li Y (2020) Semi-supervised learning framework based on statistical analysis for image set classification. Pattern Recognit 107:107500
Liu Z, Huang S, Jin W, Mu Y (2021) Broad learning system for semi-supervised learning. Neurocomputing 444:38–47
Jian M, Jung C (2021) Semi-supervised kernel matrix learning using adaptive constraint-based seed propagation. Pattern Recognit 112:107750
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436
Peng H, Du B, Liu M, Liu M, Ji S, Wang S, Zhang X, He L (2021) Dynamic graph convolutional network for long-term traffic flow prediction with reinforcement learning. Inform Sci 578:401–416
Zheng C, Fan X, Wang C, Qi J (2020) GMAN: A graph multi-attention network for traffic prediction. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, pp 1234–1241. AAAI Press
Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations
Yuan Y, Mou L, Lu X (2015) Scene recognition by manifold regularized deep learning architecture. IEEE Trans Neural Netw Learn Syst 26(10):2222–2233
Kong D, Ding CH, Huang H, Nie F (2012) An iterative locally linear embedding algorithm. arXiv preprint http://arxiv.org/abs/1206.6463arXiv:1206.6463
Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Adv Neural Inform Process Syst 30:1024–1034
Velickovic P, Fedus W, Hamilton WL, Lio P, Bengio Y, Hjelm RD (2019) Deep graph infomax. In: International Conference on Learning Representations
Kejani MT, Dornaika F, Talebi H (2020) Graph convolution networks with manifold regularization for semi-supervised learning. Neural Netw 127:160–167
Abu-El-Haija S, Perozzi B, Kapoor A, Harutyunyan H, Alipourfard N, Lerman K, Steeg GV, Galstyan A (2019) Mixhop: higher-order graph convolutional architectures via sparsified neighborhood mixing. In: International Conference on Machine Learning
Ding X, Xia C, Zhang X, Chu X, Han J, Ding G (2021) Repmlp: re-parameterizing convolutions into fully-connected layers for image recognition
Thiede EH, Zhou W, Kondor R (2022) Autobahn: automorphism-based graph neural nets
Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International Conference on Machine Learning
Dornaika F, El Traboulsi Y (2016) Learning flexible graph-based semi-supervised embedding. IEEE Trans Cybernetics 46(1):206–218
Wang F, Zhang C (2008) Label propagation through linear neighborhoods. IEEE Trans Knowl Data Eng 20(1):55–67
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Belkin M, Niyogi P (2002) Laplacian eigenmaps and spectral techniques for embedding and clustering. Adv Neural Inf Process Syst 14:585–591
Liu W, Chang S-F (2009) Robust multi-class transductive learning with graphs. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 381–388. IEEE
Yu G, Zhang G, Domeniconi C, Yu Z, You J (2012) Semi-supervised classification based on random subspace dimensionality reduction. Pattern Recognit 45:1119–1135
Cevikalp H, Verbeek JJ, Jurie F, Kläser A (2008) Semi-supervised dimensionality reduction using pairwise equivalence constraints. Int Conf Comput Vis Theory Appl 1:489–496
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Dornaika, F., Hoang, V.T. Deep data representation with feature propagation for semi-supervised learning. Int. J. Mach. Learn. & Cyber. 14, 1303–1316 (2023). https://doi.org/10.1007/s13042-022-01701-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-022-01701-9