Abstract
Extreme learning machines are used for various contexts in artificial intelligence, such as for classifying patterns, performing time series prediction and regression problems, and being a more viable solution for training hidden layer weights to determine values of the learning model. However, the essence, the model determines that these weights should be determined randomly, and the Moore Penrose pseudoinverse will define only the weights that will act in the output layer. Random weights make this learning a black box because there is no relationship between the hidden layer weights and the problem data. This paper proposes the initialization of weights and bias in the hidden layer through the Wavelets transform that allows the two parameters, previously initialized at random, to be more representative about the problem domain, allowing the frequency range of the input patterns of the network to aid in the definition of weights of the ELM hidden layer. To assist in the representativeness of the data, a technique of selection of characteristics based on automatic relevance determination will be applied to the selection of the most characteristic dimensions of the problem. To compose the network structure, activation functions of the type rectified linear unit, also called ReLU, were used. The proposed model was submitted to the classification test of binary patterns in real classes, and the results show that the proposition of this work assists in bringing better accuracy to the classification results, and thus can be considered a feasible proposition to the training of neural networks that use extreme learning machine.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Avci, E., Coteli, R.: A new automatic target recognition system based on wavelet extreme learning machine. Expert Syst. Appl. 39(16), 12340–12348 (2012)
Bache, K., Lichman, M.: UCI machine learning repository (2013)
de Campos Souza, P.V., Araujo, V.S., Guimaraes, A.J., Araujo, V.J.S., Rezende, T.S.: Method of pruning the hidden layer of the extreme learning machine based on correlation coefficient. In: 2018 IEEE Latin American Conference on Computational Intelligence (LA-CCI), pp. 1–6, November 2018. https://doi.org/10.1109/LA-CCI.2018.8625247
Cao, J., Lin, Z., Huang, G.B.: Composite function wavelet neural networks with extreme learning machine. Neurocomputing 73(7–9), 1405–1416 (2010)
Chacko, B.P., Krishnan, V.V., Raju, G., Anto, P.B.: Handwritten character recognition using wavelet energy and extreme learning machine. Int. J. Mach. Learn. Cybern. 3(2), 149–161 (2012)
Daubechies, I.: The wavelet transform, time-frequency localization and signal analysis. IEEE Trans. Inf. Theory 36(5), 961–1005 (1990)
Deo, R.C., Tiwari, M.K., Adamowski, J.F., Quilty, J.M.: Forecasting effective drought index using a wavelet extreme learning machine (W-ELM) model. Stochast. Environ. Res. Risk Assess. 31(5), 1211–1240 (2017)
Ding, S., Zhang, J., Xu, X., Zhang, Y.: A wavelet extreme learning machine. Neural Comput. Appl. 27(4), 1033–1040 (2016)
Golub, G., Kahan, W.: Calculating the singular values and pseudo-inverse of a matrix. J. Soc. Ind. Appl. Math. Ser. B: Numer. Anal. 2(2), 205–224 (1965)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1–3), 489–501 (2006)
Javed, K., Gouriveau, R., Zerhouni, N.: SW-ELM: a summation wavelet extreme learning machine algorithm with a priori parameter initialization. Neurocomputing 123, 299–307 (2014)
Karlik, B., Olgac, A.V.: Performance analysis of various activation functions in generalized MLP architectures of neural networks. Int. J. Artif. Intell. Expert Syst. 1(4), 111–122 (2011)
Kuang, Y., Wu, Q., Shao, J., Wu, J., Wu, X.: Extreme learning machine classification method for lower limb movement recognition. Cluster Comput. 20(4), 3051–3059 (2017)
Li, B., Cheng, C.: Monthly discharge forecasting using wavelet neural networks with extreme learning machine. Sci. China Technol. Sci. 57(12), 2441–2452 (2014)
Li, R., Wang, X., Lei, L., Song, Y.: \(l\_\{21\}\)-norm based loss function and regularization extreme learning machine. IEEE Access 7, 6575–6586 (2019)
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol. 30, p. 3 (2013)
Martínez-Martínez, J.M., Escandell-Montero, P., Soria-Olivas, E., Martín-Guerrero, J.D., Magdalena-Benedito, R., Gómez-Sanchis, J.: Regularized extreme learning machine for regression problems. Neurocomputing 74(17), 3716–3721 (2011)
McDonnell, M.D., Tissera, M.D., Vladusich, T., Van Schaik, A., Tapson, J.: Fast, simple and accurate handwritten digit classification by training shallow neural network classifiers with the ‘extreme learning machine’ algorithm. PLoS ONE 10(8), e0134254 (2015)
Miche, Y., Sorjamaa, A., Bas, P., Simula, O., Jutten, C., Lendasse, A.: OP-ELM: optimally pruned extreme learning machine. IEEE Trans. Neural Netw. 21(1), 158–162 (2010)
Neal, R.M.: Bayesian Learning for Neural Networks, vol. 118. Springer, Heidelberg (2012)
Peck, C.C., Sheiner, L.B., Nichols, A.I.: The problem of choosing weights in nonlinear regression analysis of pharmacokinetic data. Drug Metab. Rev. 15(1–2), 133–148 (1984)
Pinto, D., Lemos, A.P., Braga, A.P., Horizonte, B., Gerais-Brazil, M.: An affinity matrix approach for structure selection of extreme learning machines. In: Proceedings, p. 343. Presses universitaires de Louvain (2015)
Wipf, D.P., Nagarajan, S.S.: A new view of automatic relevance determination. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S.T. (eds.) Advances in Neural Information Processing Systems 20, pp. 1625–1632. Curran Associates, Inc. (2008). http://papers.nips.cc/paper/3372-a-new-view-of-automatic-relevance-determination.pdf
Zeng, Y., Xu, X., Fang, Y., Zhao, K.: Traffic sign recognition using deep convolutional networks and extreme learning machine. In: He, X., et al. (eds.) IScIDE 2015. LNCS, vol. 9242, pp. 272–280. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23989-7_28
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
de Campos Souza, P.V., Araujo, V.J.S., Araujo, V.S., Batista, L.O., Guimaraes, A.J. (2019). Pruning Extreme Wavelets Learning Machine by Automatic Relevance Determination. In: Macintyre, J., Iliadis, L., Maglogiannis, I., Jayne, C. (eds) Engineering Applications of Neural Networks. EANN 2019. Communications in Computer and Information Science, vol 1000. Springer, Cham. https://doi.org/10.1007/978-3-030-20257-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-20257-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20256-9
Online ISBN: 978-3-030-20257-6
eBook Packages: Computer ScienceComputer Science (R0)