A pruning ensemble model of extreme learning machine with $L_{1/2}$ regularizer

Abstract

Extreme learning machine (ELM) as an emerging branch of machine learning has shown its good generalization performance at a very fast learning speed. Nevertheless, the preliminary ELM and other evolutional versions based on ELM cannot provide the optimal solution of parameters between the hidden and output layer and cannot determine the suitable number of hidden nodes automatically. In this paper, a pruning ensemble model of ELM with $L_{1/2}$ regularizer (PE-ELMR) is proposed to solve above problems. It involves two stages. First, we replace the original solving method of the output parameter in ELM to a minimum squared-error problem with sparse solution by combining ELM with $L_{1/2}$ regularizer. Second, in order to get the required minimum number for good performance, we prune the nodes in hidden layer with the ensemble model, which reflects the superiority in searching the reasonable hidden nodes. Experimental results present the good performance of our method PE-ELMR, compared with ELM, OP-ELM and PE-ELMR (L1), for regression and classification problems under a variety of benchmark datasets.