Abstract
For solving the problem that extreme learning machine (ELM) algorithm uses fixed activation function and cannot be residual compensation, a new learning algorithm called variable activation function extreme learning machine based on residual prediction compensation is proposed. In the learning process, the proposed method adjusts the steep degree, position and mapping scope simultaneously. To enhance the nonlinear mapping capability of ELM, particle swarm optimization algorithm is used to optimize variable parameters according to root-mean square error for the prediction accuracy of the mode. For further improving the predictive accuracy, the auto-regressive moving average model is used to model the residual errors between actual value and predicting value of variable activation function extreme learning machine (V-ELM). The prediction of residual errors is used to rectify the prediction value of V-ELM. Simulation results verified the effectiveness and feasibility of this method by using Pole, Auto-Mpg, Housing, Diabetes, Triazines and Stock benchmark datasets. Also, it was implemented to develop a soft sensor model for the gasoline dry point in delayed coking and some satisfied results were obtained.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chacko BP, Krishnan VRV, Raju G, Anto PB (2011) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern. doi:10.1007/s13042-011-0049-5
Chang Y-Q, Li Y-C, Wang F-L et al (2007) Soft sensing modeling based on extreme learning machine for biochemical processes. J Syst Simul 19(23):5587–8890
Hu YG, Li W, Hu JM (2004) An artificial neural network with improved activation function and its application. Geomat Inf Sci Wuhan Univ 29(10):916–919
Huang G-B, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062
Huang G-B, Siew C-K (2004) Extreme learning machine: RBF network case. In: 8th International Conference on control, automation, robotics and vision, pp 1029–1036
Huang G-B, Zhu Q-Y, Siew C-K (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of International Joint Conference on neural networks, vol 70, pp 25–29
Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501
Huang G-B, Li M-B, Chen L, Siew C-K (2008) Incremental extreme learning machine with fully complex hidden nodes. Neurocomputing 71:576–583
Huang G-B, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122
Huynh HT, Won Y (2008) Extreme learning machine with fuzzy activation function. In: Fifth International Joint Conference on INC, IMS and IDC, pp 303–307
Jun W, Wang S, Chung F (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cybern 2(4):261–271
Kennedy J, Eberhert R (1995) Particle swarm optimization. In: IEEE International Conference on neural networks, pp 192–194
Lan Y, Lan Y, Soh YC, Huang G-B (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72:3391–3395
Liang NY, Huang G-B, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423
Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2009) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):1–5
Nizar AH, Dong ZY, Wang Y (2008) Power utility nontechnical loss analysis with extreme learning machine method. IEEE Trans Power Syst 23(3):946–955
Sun Z-L, Choi T-M (2008) Sales forecasting using extreme learning machine with applications in fashion retailing. Decis Support Syst 46:411–419
Tian H-X, Mao Z-Z, Wang J-Z (2008) Hybrid modeling based on ELM for soft sensing of end temperature of molten steel in LF. J Northeastern Univ (Nat Sci) 29(1):33–36
Wang HY, Lu S (2006) Nonlinear time series analysis and its application. Science Press, Beijing
Wang GT, Li P, Su CL (2010) Soft sensor modeling based on multiple extreme learning machine. In: The International Conference on Electrical Engineering and Automatic Control, vol 2, pp 916–919
Wang X-Z, Chen A-X, Feng H-M (2011) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525
Zhu Q-Y, Qin AK, Suganthan PN, Huang G-B (2005) Evolutionary extreme learning machine. Pattern Recognit 38(10):1759–1763
Acknowledgments
This work was supported by the Program for Liaoning Science and Technology Innovative Research of China (2007T103) and Program for Liaoning Excellent Talents in University of China (2008RC32). The project was sponsored by “Liaoning BaiQianWan Talents program” of China (2008921039).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, Gt., Li, P. & Cao, Jt. Variable activation function extreme learning machine based on residual prediction compensation. Soft Comput 16, 1477–1484 (2012). https://doi.org/10.1007/s00500-012-0817-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-012-0817-5