Skip to main content
SpringerLink
Log in

Adaptive multiple graph regularized semi-supervised extreme learning machine

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Semi-supervised extreme learning machine (SSELM) was proposed as an effective algorithm for machine learning and pattern recognition. However, the performance of SSELM heavily depends on whether the underlying geometrical structure of the data can be well exploited. Though many techniques have been utilized for constructing graph to represent the data structure, which of them can best reflect the intrinsic distribution of complicated input data is still needed to be verified. Aiming to solve this problem, we propose a novel algorithm called adaptive multiple graph regularized semi-supervised extreme learning machine (AMGR-SSELM). The contributions of the proposed algorithm are as follows: (1) AMGR-SSELM employs multiple graph structures extracted from training samples to characterize the structure of input data. Since these graphs are constructed based on different principles and complementary with each other, the underlying data distribution can be well exploited through combining them. (2) A nonnegative weight vector is introduced into AMGR-SSELM to adaptively combine the multiple graphs for representing different data. (3) An explicit classifier can be learnt in our algorithm, which overcomes the ‘out of sample’ problem. (4) A simple and efficient iterative update approach is also proposed to optimize AMGR-SSELM. In addition, we compare the proposed approach with other classification methods and some extreme learning machine variants on five benchmark image databases (Yale, Extended YaleB, CMU PIE, AR and FKP). The results of extensive experiments show the advantages and effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • An S, Liu W, Venkatesh S (2007) Face recognition using kernel ridge regression. In: Proceeding of IEEE international conference on computer vision

  • Bengio Y, Paiement J, Vincent P (2003) Out-of-sample extensions for LLE, isomap, MDS, eigenmaps and spectral clustering. In: Proceedings of advances in neural information processing systems, pp 177–184

  • Boyd S, Vandenberghe L (2009) Convex optimization. Cambridge University Press, New York

    MATH  Google Scholar 

  • Cai X, Nie X et al (2013) Heterogeneous image features integration via multi-modal semi-supervised learning model. In: 2013 IEEE international conference on computer vision (ICCV). IEEE, pp 1737–1744

  • Cambria E et al (2013) Extreme learning machines trends & controversies. IEEE Intell Syst 28(6):30–59

    Article  Google Scholar 

  • Cao J et al (2016) Extreme learning machine and adaptive sparse representation for image classification. Neural Netw 81:91–102

    Article  Google Scholar 

  • Cheng B, Yang J et al (2010) Learning with 11-graph for image analysis. IEEE Trans Image Process 19(4):858–866

    Article  MathSciNet  MATH  Google Scholar 

  • Deng W et al (2016) A fast SVD-hidden-nodes based extreme learning machine for large-scale data analytics. Neural Netw 77:14–28

    Article  Google Scholar 

  • Ding S, Zhang N et al (2017) Unsupervised extreme learning machine with representational features. Int J Mach Learn Cybern 8(2):587–595

    Article  Google Scholar 

  • Duda R, Hart P, Stork D (2012) Pattern classification. Wiley, New York

    MATH  Google Scholar 

  • Gastaldo P et al (2016) SIM-ELM: connecting the elm model with similarity-function learning. Neural Netw 74:22–34

    Article  Google Scholar 

  • He X, Niyogi P (2005) Locality preserving projections. Adv Neural Inf Process Syst 16(1):186–197

    Google Scholar 

  • Huang G, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16):3460–3468

    Article  Google Scholar 

  • Huang G, Chen L, Siew C (2006a) Universal approximation using incremental constructive feed forward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  • Huang G, Zhu Q, Siew C (2006b) Extreme learning machine: theory and applications. Neurocomputing 70(1):489–501

    Article  Google Scholar 

  • Huang G et al (2015a) Trends in extreme learning machines: a review. Neural Netw 61:32–48

    Article  MATH  Google Scholar 

  • Huang G, Bai Z et al (2015b) Local receptive fields based extreme learning machine. IEEE Comput Intell Mag 10(2):18–29

    Article  Google Scholar 

  • Iosifidis A, Tefas A, Pitas I (2015) Graph embedded extreme learning machine. IEEE Trans Cybern 46(1):311–324

    Article  Google Scholar 

  • Karasuyama M, Mamitsuka H (2013) Multiple graph label propagation by sparse integration. IEEE Trans Neural Netw Learn Syst 24(12):1999–2012

    Article  Google Scholar 

  • Kasun L, Yang Y et al (2016) Dimension reduction with extreme learning machine. IEEE Trans Image Process 25:3906–3918

    Article  MathSciNet  Google Scholar 

  • Lee K, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698

    Article  Google Scholar 

  • Li P, Bu J et al (2013) Relational multimanifold coclustering. IEEE Trans Cybern 43(6):1871–1881

    Article  Google Scholar 

  • Liang N, Huang G, Saratchandran P (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17:1411–1423

    Article  Google Scholar 

  • Liu G, Lin Z, Yu Y (2010) Robust subspace segmentation by low-rank representation. In: International conference on machine learning, pp 663–670

  • Liu G, Lin Z et al (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Article  MathSciNet  Google Scholar 

  • Liu M, Liu B et al (2017) Semi-supervised low rank kernel learning algorithm via extreme learning machine. Int J Mach Learn Cybern 8(3):1039–1052

    Article  Google Scholar 

  • Lu C, Min H et al (2012) Robust and efficient subspace segmentation via least squares regression. In: European conference on computer vision, pp 347–360

  • Mao W, Wang J, Xue Z (2017) An ELM-based model with sparse-weighting strategy for sequential data imbalance problem. Int J Mach Learn Cybern 8(4):1333–1345

    Article  Google Scholar 

  • Martinez A, Benavente R (1998) The AR face database, CVC technical report, vol 24

  • Naseem I, Togneri R, Bennamoun M (2010) Linear regression for face recognition. IEEE Trans Pattern Anal Mach Intell 32(11):2106–2112

    Article  Google Scholar 

  • Peng Y, Wang S et al (2015) Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing 149:340–353

    Article  Google Scholar 

  • Peng X et al (2017) Constructing the L2-graph for robust subspace learning and subspace clustering. IEEE Trans Cybern 47(4):1053–1066

    Article  Google Scholar 

  • Qiao S, Tang C et al (2010) PutMode: prediction of uncertain trajectories in moving objects databases. Appl Intell 33(3):370–386

    Article  Google Scholar 

  • Qiao S, Han N et al (2015a) TraPlan: an effective three-in-one trajectory-prediction model in transportation networks. IEEE Trans Intell Transp Syst 16(3):1188–1198

    Article  Google Scholar 

  • Qiao S, Shen D et al (2015b) A self-adaptive parameter selection trajectory prediction approach via hidden Markov models. IEEE Trans Intell Transp Syst 16(1):284–296

    Article  Google Scholar 

  • Qiao S, Han N et al (2017) Predicting long-term trajectories of connected vehicles via the prefix-projection technique. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2017.2750075

    Google Scholar 

  • Qiao S, Han N et al (2018) A fast parallel community discovery model on complex networks through approximate optimization. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2018.2803818

    Google Scholar 

  • Remmert R (2012) Theory of complex functions. Springer, Berlin

    Google Scholar 

  • Rong H, Huang G, Sundararajan N (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Trans Syst Man Cybern B 39:1067–1072

    Article  Google Scholar 

  • Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  • Rumelhart D, Hinton G, Williams R (1986) Learning representations by back-propagating errors. Nature 323(9):533–536

    Article  MATH  Google Scholar 

  • Salaken S et al (2017) Extreme learning machine based transfer learning algorithms: a survey. Neurocomputing 267:516–524

    Article  Google Scholar 

  • Tang J, Deng C, Huang G (2016) Extreme learning machine for multilayer perceptron. IEEE Trans Neural Netw Learn Syst 27(4):809–821

    Article  MathSciNet  Google Scholar 

  • Terence S, Simon B, Maan B (2003) The CMU pose, illumination, and expression (PIE) database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618

    Article  Google Scholar 

  • Wang J, Bensmail H, Gao X (2013) Multiple graph regularized nonnegative matrix factorization. Pattern Recognit 46(10):2840–2847

    Article  MATH  Google Scholar 

  • Wang Z et al (2017) Kernel fusion based extreme learning machine for cross-location activity recognition. Inf Fusion 37:1–9

    Article  Google Scholar 

  • Wright J, Yang AY et al (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  • Yale University Face Database (2002). http://cvc.yale.edu/projects/yalefaces/yalefaces.html. Accessed 10 Dec 2002

  • Yang B, Chen S (2010) Sample-dependent graph construction with application to dimensionality reduction. Neurocomputing 74(1):301–314

    Article  Google Scholar 

  • Yang L, Yang S et al (2017) Incremental Laplacian regularization extreme learning machine for online learning. Appl Soft Comput 59:546–555

    Article  Google Scholar 

  • Yao L, Ge Z (2018) Deep learning of semi-supervised process data with hierarchical extreme learning machine and soft sensor application. IEEE Trans Ind Electron 65(2):1490–1498

    Article  Google Scholar 

  • Yi Y, Zhou W et al (2014) Face recognition using spatially smoothed discriminant structure-preserved projections. J Electron Imaging 23(2):023012-1-20

    Article  Google Scholar 

  • Yi Y, Bi C et al (2015a) Semi-supervised local ridge regression for local matching based face recognition. Neurocomputing 167:132–146

    Article  Google Scholar 

  • Yi Y, Shi Y et al (2015b) Label propagation based semi-supervised non-negative matrix factorization for feature extraction. Neurocomputing 149:1021–1037

    Article  Google Scholar 

  • Yu J, Wang M et al (2012) Semi-supervised multiview distance metric learning for cartoon synthesis. IEEE Trans Image Process 21(11):4636–4648

    Article  MathSciNet  MATH  Google Scholar 

  • Zhai J, Zhang S, Wang C (2017) The classification of imbalanced large data sets based on mapreduce and ensemble of elm classifiers. Int J Mach Learn Cybern 8(3):1009–1017

    Article  Google Scholar 

  • Zhang L, Zhang D (2016) Robust visual knowledge transfer via extreme learning machine-based domain adaptation. IEEE Trans Image Process 25(10):4959–4973

    Article  MathSciNet  Google Scholar 

  • Zhang L, Zhang D (2017) Evolutionary cost-sensitive extreme learning machine. IEEE Trans Neural Netw Learn Syst 28(12):3045–3060

    Article  MathSciNet  Google Scholar 

  • Zhang L, Zhang L et al (2010) Online finger-knuckle-print verification for personal authentication. Pattern Recognit 43(7):2560–2571

    Article  MATH  Google Scholar 

  • Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: which helps face recognition? In: Proceeding of IEEE international conference on computer vision, pp 471–478

  • Zhang Z, Zhao M, Chow TWS (2015) Graph based constrained semi-supervised learning framework via label propagation over adaptive neighborhood. IEEE Trans Knowl Data Eng 27(9):2362–2376

    Article  Google Scholar 

  • Zhang B et al (2018) Ensemble based reactivated regularization extreme learning machine for classification. Neurocomputing 275:255–266

    Article  Google Scholar 

  • Zhou D et al (2004) Learning with local and global consistency. Adv Neural Inf Process Syst 16:321–328

    Google Scholar 

  • Zhou Y, Liu B et al (2015) Semi-supervised extreme learning machine with manifold and pairwise constraints regularization. Neurocomputing 149(PA):180–186

    Article  Google Scholar 

Download references

Acknowledgements

This study was supported in part by the National Natural Science Foundation of China under Grants Nos. 61602221, 61772091, 61672150, 61602222 and 61702092; the Natural Science Foundation of Jiangxi Province under Grant No. 20171BAB212009; the Planning Foundation for Humanities and Social Sciences of Ministry of Education of China under Grant No. 15YJAZH058; the Innovative Research Team Construction Plan in Universities of Sichuan Province under Grant No. 18TD0027; the Scientific Research Foundation for Advanced Talents of Chengdu University of Information Technology under Grant Nos. KYTZ201715 and KYTZ201750; the Scientific Research Foundation for Young Academic Leaders of Chengdu University of Information Technology under Grant No. J201701.

Author information

Authors and Affiliations

  1. School of Software, Jiangxi Normal University, Nanchang, 330022, Jiangxi, China

    Yugen Yi & Qinghua Liu

  2. School of Cybersecurity, Chengdu University of Information Technology, Chengdu, 610255, Sichuan, China

    Shaojie Qiao

  3. Key Laboratory of Sichuan Province, Chengdu University of Information Technology, Chengdu, 610255, Sichuan, China

    Shaojie Qiao

  4. College of Information Science and Engineering, Northeastern University, Shenyang, 110004, Liaoning, China

    Wei Zhou

  5. School of Computer Science and Information Technology, Northeast Normal University, Changchun, 130117, Jilin, China

    Caixia Zheng & Jianzhong Wang

  6. Key Laboratory of Applied Statistics of MOE, Northeast Normal University, Changchun, 130117, Jilin, China

    Caixia Zheng & Jianzhong Wang

Authors
  1. Yugen Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaojie Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Caixia Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qinghua Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianzhong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shaojie Qiao or Jianzhong Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Communicated by X. Wang, A.K. Sangaiah, M. Pelillo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yi, Y., Qiao, S., Zhou, W. et al. Adaptive multiple graph regularized semi-supervised extreme learning machine. Soft Comput 22, 3545–3562 (2018). https://doi.org/10.1007/s00500-018-3109-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-018-3109-x

Keywords

Search

Navigation