An Improved Evolutionary Extreme Learning Machine Based on Multiobjective Particle Swarm Optimization

Abstract

Extreme learning machine (ELM) proposed for training single-hidden-layer feedforward neural network has drawn much attention. Since ELM randomly initializes the hidden nodes and analytically determines the output weights, it may easily lead to redundant hidden representation and numerical instability problem. Conventional single-objective optimization methods have been applied for improving ELM, however multiple objectives associate with ELM should be considered. This paper proposes an evolutionary ELM method based on multiobjective particle swam optimization to overcome the drawbacks. The input weights and hidden biases are learnt by optimizing three objectives simultaneously including the root mean squared error, the norm of the output weights and the sparsity of hidden output matrix. The proposed method tends to select small weights which make ELM becomes robust to small input changes for improving the numerical stability. Moreover, this algorithm inhibits activated neurons and enables ELM to obtain an informative hidden representation. The proposed approach can achieve good generalization performance with sparse hidden representation. Experimental results verify that the new algorithm is highly competitive and superior to ELM and conventional evolutionary ELMs on benchmark classification problems.