Skip to main content

Advertisement

Log in

Deep data representation with feature propagation for semi-supervised learning

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Graph-based embedding has attracted much attention in the fields of machine learning and pattern recognition. It is becoming an indispensable tool for data representation. It can be useful for all types of learning: unsupervised, semi-supervised, and supervised. In this correspondence, we present a graph-based, deep and flexible method for data representation with feature propagation. The presented framework ensures several desired features such as graph-based regularization, a flexible embedding model, graph-based feature propagation, and a deep learning architecture. The model can be learned layer by layer. In each layer, the nonlinear data representation and the unknown convolved data based regression are jointly estimated with a closed-form solution. We evaluate the proposed system on semi-supervised classification tasks using six public image datasets. These experiments demonstrate the effectiveness of the presented framework, which compares favorably to many competing semi-supervised approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The data that support the findings of this study are available from the corresponding author, F. D., upon request.

References

  1. Zhu R, Dornaika F, Ruichek Y (2019) Joint graph based embedding and feature weighting for image classification. Pattern Recognit 93:458–469

    Article  MATH  Google Scholar 

  2. Zhu R, Dornaika F, Ruichek Y (2019) Learning a discriminant graph-based embedding with feature selection for image categorization. Neural Netw 111:35–46

    Article  MATH  Google Scholar 

  3. Han C, Zhou D, Xie Y, Lei Y, Shi J (2021) Label propagation with multi-stage inference for visual domain adaptation. Knowl-Based Syst 216:106809

    Article  Google Scholar 

  4. He F, Nie F, Wang R, Jia W, Zhang F, Li X (2020) Semisupervised band selection with graph optimization for hyperspectral image classification. IEEE Trans Geosci Remote Sensing 59:10298–10311

    Article  Google Scholar 

  5. Zhao M, Zhang Y, Zhang Z, Liu J, Kong W (2019) Alg: adaptive low-rank graph regularization for scalable semi-supervised and unsupervised learning. Neurocomputing 370:16–27

    Article  Google Scholar 

  6. Jia J, Ruan Q, Jin Y, An G, Ge S (2020) View-specific subspace learning and re-ranking for semi-supervised person re-identification. Pattern Recognit 108:107568

    Article  Google Scholar 

  7. Yang Y, Zhan D-C, Wu Y-F, Liu Z-B, Xiong H, Jiang Y (2021) Semi-supervised multi-modal clustering and classification with incomplete modalities. IEEE Trans Knowl Data Eng 33(2):682–695

    Google Scholar 

  8. Zhu R, Dornaika F, Ruichek Y (2021) Inductive semi-supervised learning with graph convolution based regression. Neurocomputing 434:315–322

    Article  Google Scholar 

  9. Fan D-P, Zhou T, Ji G-P, Zhou Y, Chen G, Fu H, Shen J, Shao L (2020) Inf-net: automatic COVID-19 lung infection segmentation from ct images. IEEE Trans Med Imaging 39(8):2626–2637

    Article  Google Scholar 

  10. Hamid M (2019) Semi-supervised learning for plankton image classification. Master’s thesis, Lappeenranta-Lahti University of Technology LUT

  11. Zhu R, Dornaika F, Ruichek Y (2020) Semi-supervised elastic manifold embedding with deep learning architecture. Pattern Recognit 107:107425

    Article  Google Scholar 

  12. An J, Zhao X, Shi M, Liu X, Guo J (2021) Joint neighborhood preserving and projected clustering for feature extraction. Neurocomputing 488:572–580

    Article  Google Scholar 

  13. Dornaika F, Baradaaji A, El Traboulsi Y (2021) Semi-supervised classification via simultaneous label and discriminant embedding estimation. Inform Sci 546:146–165

    Article  MathSciNet  MATH  Google Scholar 

  14. Hu Y, You H, Wang Z, Wang Z, Zhou E, Gao Y (2021) Graph-mlp: mode classification without message passing in graph

  15. Jiang B, Zhang Z, Lin D, Tang J, Luo B (2019) Semi-supervised learning with graph learning-convolutional networks. In: IEEE/CVF conference on computer vision and pattern recognition, pp 11313–11320

  16. Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. Adv Neural Inform Process Syst 16:321–328

    Google Scholar 

  17. Zhu X, Ghahramani Z, Lafferty JD (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: International conference on machine learning, pp 912–919

  18. Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: IEEE International Conference on Computer Vision, pp 1–7

  19. Chen H-T, Chang H-W, Liu T-L (2005) Local discriminant embedding and its variants. IEEE Conf Comput Vision Pattern Recognit 2:846–853

    Google Scholar 

  20. Hou C, Nie F, Li X, Yi D, Wu Y (2014) Joint embedding learning and sparse regression: a framework for unsupervised feature selection. IEEE Trans Cybernetics 44(6):793–804

    Article  Google Scholar 

  21. Nie F, Cai G, Li J, Li X (2018) Auto-weighted multi-view learning for image clustering and semi-supervised classification. IEEE Trans Image Process 27(3):1501–1511

    Article  MathSciNet  MATH  Google Scholar 

  22. Nie F, Xu D, Tsang IW-H, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932

    Article  MathSciNet  MATH  Google Scholar 

  23. Dornaika F, El Traboulsi Y (2017) Margin based semi-supervised elastic embedding for face image analysis. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1313–1320

  24. El Traboulsi Y, Dornaika F, Assoum A (2015) Kernel flexible manifold embedding for pattern classification. Neurocomputing 167:517–527

    Article  MATH  Google Scholar 

  25. Nie F, Cai G, Li X (2017) Multi-view clustering and semi-supervised classification with adaptive neighbours. In: Thirty-First AAAI Conference on Artificial Intelligence

  26. Dornaika F, Traboulsi YE (2019) Joint sparse graph and flexible embedding for graph-based semi-supervised learning. Neural Netw 114:91–95

    Article  Google Scholar 

  27. Nie F, Wang Z, Wang R, Li X (2020) Submanifold-preserving discriminant analysis with an auto-optimized graph. IEEE Trans Cybernet 50(8):3682–3695

    Article  Google Scholar 

  28. Nie F, Dong X, Li X (2020) Unsupervised and semisupervised projection with graph optimization. IEEE Trans Neural Netw Learn Syst 32:1547–1559

    Article  MathSciNet  Google Scholar 

  29. Nie F, Wang Z, Wang R, Li X (2021) Adaptive local embedding learning for semi-supervised dimensionality reduction. IEEE Trans Knowl Data Eng 34:4609–4621

    Article  Google Scholar 

  30. Ziraki N, Dornaika F, Bosaghzadeh A (2022) Multiple-view flexible semi-supervised classification through consistent graph construction and label propagation. Neural Netw 146:174–180

    Article  Google Scholar 

  31. Wang F, Zhu L, Xie L, Zhang Z, Zhong M (2021) Label propagation with structured graph learning for semi-supervised dimension reduction. Knowl-Based Syst 225:107130

    Article  Google Scholar 

  32. Kang Z, Peng C, Cheng Q, Liu X, Peng X, Xu Z, Tian L (2021) Structured graph learning for clustering and semi-supervised classification. Pattern Recognit 110:107627

    Article  Google Scholar 

  33. Yan W, Sun Q, Sun H, Li Y (2020) Semi-supervised learning framework based on statistical analysis for image set classification. Pattern Recognit 107:107500

    Article  Google Scholar 

  34. Liu Z, Huang S, Jin W, Mu Y (2021) Broad learning system for semi-supervised learning. Neurocomputing 444:38–47

    Article  Google Scholar 

  35. Jian M, Jung C (2021) Semi-supervised kernel matrix learning using adaptive constraint-based seed propagation. Pattern Recognit 112:107750

    Article  Google Scholar 

  36. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  37. Peng H, Du B, Liu M, Liu M, Ji S, Wang S, Zhang X, He L (2021) Dynamic graph convolutional network for long-term traffic flow prediction with reinforcement learning. Inform Sci 578:401–416

    Article  MathSciNet  Google Scholar 

  38. Zheng C, Fan X, Wang C, Qi J (2020) GMAN: A graph multi-attention network for traffic prediction. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, pp 1234–1241. AAAI Press

  39. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations

  40. Yuan Y, Mou L, Lu X (2015) Scene recognition by manifold regularized deep learning architecture. IEEE Trans Neural Netw Learn Syst 26(10):2222–2233

    Article  MathSciNet  Google Scholar 

  41. Kong D, Ding CH, Huang H, Nie F (2012) An iterative locally linear embedding algorithm. arXiv preprint http://arxiv.org/abs/1206.6463arXiv:1206.6463

  42. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Adv Neural Inform Process Syst 30:1024–1034

    Google Scholar 

  43. Velickovic P, Fedus W, Hamilton WL, Lio P, Bengio Y, Hjelm RD (2019) Deep graph infomax. In: International Conference on Learning Representations

  44. Kejani MT, Dornaika F, Talebi H (2020) Graph convolution networks with manifold regularization for semi-supervised learning. Neural Netw 127:160–167

    Article  Google Scholar 

  45. Abu-El-Haija S, Perozzi B, Kapoor A, Harutyunyan H, Alipourfard N, Lerman K, Steeg GV, Galstyan A (2019) Mixhop: higher-order graph convolutional architectures via sparsified neighborhood mixing. In: International Conference on Machine Learning

  46. Ding X, Xia C, Zhang X, Chu X, Han J, Ding G (2021) Repmlp: re-parameterizing convolutions into fully-connected layers for image recognition

  47. Thiede EH, Zhou W, Kondor R (2022) Autobahn: automorphism-based graph neural nets

  48. Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International Conference on Machine Learning

  49. Dornaika F, El Traboulsi Y (2016) Learning flexible graph-based semi-supervised embedding. IEEE Trans Cybernetics 46(1):206–218

    Article  Google Scholar 

  50. Wang F, Zhang C (2008) Label propagation through linear neighborhoods. IEEE Trans Knowl Data Eng 20(1):55–67

    Article  Google Scholar 

  51. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  52. Belkin M, Niyogi P (2002) Laplacian eigenmaps and spectral techniques for embedding and clustering. Adv Neural Inf Process Syst 14:585–591

    Google Scholar 

  53. Liu W, Chang S-F (2009) Robust multi-class transductive learning with graphs. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 381–388. IEEE

  54. Yu G, Zhang G, Domeniconi C, Yu Z, You J (2012) Semi-supervised classification based on random subspace dimensionality reduction. Pattern Recognit 45:1119–1135

    Article  MATH  Google Scholar 

  55. Cevikalp H, Verbeek JJ, Jurie F, Kläser A (2008) Semi-supervised dimensionality reduction using pairwise equivalence constraints. Int Conf Comput Vis Theory Appl 1:489–496

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to F. Dornaika.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dornaika, F., Hoang, V.T. Deep data representation with feature propagation for semi-supervised learning. Int. J. Mach. Learn. & Cyber. 14, 1303–1316 (2023). https://doi.org/10.1007/s13042-022-01701-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01701-9

Keywords

Navigation