A new procedure for optimization of hidden layer neurons during learning through gradient descent process of neural network and improvement of performance in the chaos forecasting

Abstract

In the General Backpropagation Neural Network (GBPN), the training process has been found to be dependent on total neurons in the hidden layer (p). The number of weights and biases of hidden and output layers depends on the value of 'p'. Two novel procedures have been suggested in the present study to select the optimum value of 'p', and have been applied in the modeling for forecasting of area-weighted rainfall (chaotic series) with four predictors. In this case, an Optimum Back-Propagation Neural Network-OBPN was obtained from GBPN where two hidden neurons (p = 2) was found to be optimum. It was trained up to global minima (400,000 epochs). During the training and testing periods, the average deviations from the actual were found to be 7.8% and 9.3%, respectively. It was forecasted successfully for the year 2016, 2017, 2018, and 2019 with 6.9%, 0.6%, 11.8%, and 4.6% deviation from the actual, respectively. It was found that the network would not necessarily perform well if the hidden layer contained a large number of neurons. For optimum performance, their number should be low but optimal. Experimental observations during its optimization process and performance are given through this research article.