Abstract
Extreme learning machine (ELM) has been proposed for solving fast supervised learning problems by applying random computational nodes in the hidden layer. Similar to support vector machine, ELM cannot handle high-dimensional data effectively. Its generalization performance tends to become bad when it deals with high-dimensional data. In order to exploit high-dimensional data effectively, a two-stage extreme learning machine model is established. In the first stage, we incorporate ELM into the spectral regression algorithm to implement dimensionality reduction of high-dimensional data and compute the output weights. In the second stage, the decision function of standard ELM model is computed based on the low-dimensional data and the obtained output weights. This is due to the fact that two stages are all based on ELM. Thus, output weights in the second stage can be approximately replaced by those in the first stage. Consequently, the proposed method can be applicable to high-dimensional data at a fast learning speed. Experimental results show that the proposed two-stage ELM scheme tends to have better scalability and achieves outstanding generalization performance at a faster learning speed than ELM.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Wang Xizhao, Chen Aixia, Feng Huimin (2011) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525
Wang Xizhao, Shao Qingyan, Qing Miao, Zhai Junhai (2013) Architecture selection for networks trained with extreme learning machine using localized generalization error model. Neurocomputing 102:3–9
Frénay B, Verleysen M (2010). Using SVMs with randomised feature spaces: an extreme learning approach. In: European Symposium on Artificial Neural Networks (ESANN): 315–320
Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162
Deng W, Zheng Q, Chen L (2009). Regularized extreme learning machine. In: IEEE Symposium on Computational Intelligence and Data Mining (CIDM):389–395
Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cyber 2(2):107–122
Wang G, Zhao Y, Wang D (2008) A protein secondary structure prediction framework based on the Extreme Learning Machine. Neurocomputing 72(1):262–268
Lan Y, Soh YC, Huang GB (2008). Extreme Learning Machine based bacterial protein subcellular localization prediction. In: IEEE International Joint Conference on Neural Networks (IJCNN):1859–1863
Mohammed AA, Minhas R, Wu Jonathan QM, Sid-Ahmed MA (2011) Human face recognition based on multidimensional PCA and extreme learning machine. Pattern Recogn 44(10):2588–2597
Wang Y, Cao F, Yuan Y (2011) A study on effectiveness of extreme learning machine. Neurocomputing 74(16):2483–2490
Cao F, Liu B, Sun Park D (2013) Image classification based on effective extreme learning machine. Neurocomputing 102:90–97
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434
Xue H, Chen S, Yang Q (2009) Discriminatively regularized least-squares classification. Pattern Recogn 42(1):93–104
Liu J, Chen Y, Liu M, Zhao Z (2011) SELM: semi-supervised ELM with application in sparse calibrated location estimation. Neurocomputing 74(16):2566–2572
Li L, Liu D, Ouyang J (2012) A new regularization classification method based on extreme learning machine in network data. J Inf Comput Sci 9(12):3351–3363
Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE T SYST MAN CY B 42(2):513–529
Cai D, He X, Han J (2007). Spectral regression for efficient regularized subspace learning. In: IEEE 11th International Conference on Computer Vision (ICCV):1–8
He X, Cai D, Shao Y, Bao H, Han J (2011) Laplacian regularized gaussian mixture model for data clustering. IEEE T KNOWL DATA EN 23(9):1406–1418
Lee KC, Ho J, Kriegman DJ (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE T PATTERN ANAL 27(5):684–698
Wu J, Wang ST, Chung FL (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cyber 2(4):261–271
Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE T PATTERN ANAL 19(7):711–720
Vink JP, de Haan G (2013). Comparison of machine learning techniques for target detection. ARTIF INTELL REV: 1–15
Acknowledgments
The authors would like to thank the editor and the anonymous reviewers for their valuable comments and constructive suggestions. This work is jointly supported by the National High Technology Research and Development Program of China (863 Program) (No. 2013AA06A411), the National Natural Science Foundation of China (No. 41302203), the Fundamental Research Funds for the Central Universities (No. 2011QNB26) and A Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, P., Huang, Y., Meng, L. et al. Two-stage extreme learning machine for high-dimensional data. Int. J. Mach. Learn. & Cyber. 7, 765–772 (2016). https://doi.org/10.1007/s13042-014-0292-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-014-0292-7