Back-propagation algorithm which varies the number of hidden units

doi:10.1016/0893-6080(91)90032-Z

Neural Networks

Volume 4, Issue 1, 1991, Pages 61-66

https://doi.org/10.1016/0893-6080(91)90032-Z Get rights and content

Abstract

This report presents a back-propagation algorithm that varies the number of hidden units. This algorithm is expected to escape local minima and makes it no longer necessary to decide the number of hidden units. We tested this algorithm on two examples. One was exclusive-OR learning and the other was 8 × 8 dot alphanumeric font learning. In both examples, the probability of becoming trapped in local minima was reduced. Furthermore, in alphanumeric font learning, the network converged two to three times faster than conventional back-propagation.

References (3)

D.E. Rumelhart et al.
Learning representations by error propagation

There are more references available in the full text version of this article.

Cited by (407)

A comprehensive review of machine learning and IoT solutions for demand side energy management, conservation, and resilient operation
2023, Energy
The energy consumption of major equipment in residential and industrial facilities can be minimized through a variety of cost-effective energy-saving measures. Most saving strategies are economically viable where several algorithms can be employed to reduce energy consumption to reduce costs to a considerable extent. Machine learning (ML) is one of these techniques. A review of recent research efforts concerning the application of ML strategies to energy conservation and management problems is presented in this study. In addition, ML approaches and strategies for energy-saving problems, management, technologies, and control methods have been discussed. A comprehensive review of all available publications is also used to make observations about past considerations. As a result, it has been concluded that ML is capable of solving a wide range of decision and management problems within a short period of time with minimal energy consumption. In addition, ML perspectives have been viewed from the perspective of emerging communication technologies, instruments, and cyber-physical systems (CPSs), along with the advancement of ultra-durable and energy-efficient Internet-of-Things (IoT) based communication sensors technology. Moreover, a comprehensive review of recent developments in ML algorithms is also included, including safe reinforcement learning (RL), Deep RL, path integral control for RL, and others not previously. Lastly, critical ML considerations such as emergency and remedial measures, integrity protection, fusion with existing robust controls, and combining preventive and emergent measures have been discussed. The implementation of recently applied ML, RL, and IoT strategies for energy management, conservation, and resilient operation is clarified in this paper. The proposed review highlights the advantages and drawbacks of the recent energy conservation strategies. Finally, the perspective solutions have been clarified to cope with the world direction for zero energy buildings.
Prediction of blood screening parameters for preliminary analysis using neural networks
2022, Predictive Modeling in Biomedical Data Mining and Analysis
Over time, the latest technologies and advancements in prediction algorithms are unmasking the innovational ways of diagnosing, predicting, and cure of diseases related to the blood. For this, artificial intelligence and neural networks have played a significant role. Monitoring blood parameters plays a vital role in detecting dormant diseases or any kind of complications. This chapter deals with the various proposed techniques used in the prediction of blood parameters. This chapter mainly presents the forecast of blood screening test features using the backpropagation neural network. The elements used in this paper are fibrinogen and globulin. The normal ranges of fibrinogen and globulin are 2–4 g/L and 20–35 g/L. The glycoprotein which circulates in the vertebrates of the human body is known as fibrinogen. Different congenital and disorders related to the acquired fibrinogen in humans can lead to dysfunctional or reduced fibrinogens. In the human body, the proteins in the blood except albumin are collectively known as globulin. Sixty percent of the proteins in the human body comprises of albumin. Various antibodies, enzymes, and all other categories of proteins fall under the globulins.
Predicting rock displacement in underground mines using improved machine learning-based models
2022, Measurement: Journal of the International Measurement Confederation
Displacement of rock mass in tunnels and underground mines is considered one of the most hazardous phenomena that can cause the collapse of the structures. In this study, the rock properties, such as the depth of the tunnels (H), the angle of rock layers (α), anti-bending moment (W_c), the width of the tunnels (b), the tensile strength of rock layers (R_n), and monitoring distance (L_b), and observation time (t), were investigated to predict rock displacement in tunnels and underground mines. Two novel soft computing models, namely Harris Hawks optimization algorithm (HHOA)-based support vector machine (SVM) model (i.e., HHOA-SVM) and Grasshopper optimization algorithm (GOA)-based SVM model (i.e., GOA-SVM), were developed for this aim based on the field measurements. A total of 12 measurement stations and 63 observations of vertical rock mass displacement, rock properties, and observation time in some underground coal mines in the Donbas region (Ukraine) were compiled as the dataset for developing soft computing models. In addition, a constraint was also added to the proposed HHOA-SVM and GOA-SVM models to prevent the model from offering negative results in predicting rock displacement. The conventional models, such as SVM (without optimization) and artificial neural network (ANN), were also investigated to compare favorably with the two proposed HHOA-SVM and GOA-SVM models. Furthermore, linear and nonlinear equations were also established to predict rock displacement and compared to the soft computing models. The results showed that the novel HHOA-SVM and GOA-SVM models provided better performances than conventional SVM and ANN models. Besides, the sensitivity of the input variables was also analyzed to discover the certain characteristics of the rock displacement phenomenon through the properties of rock and observation time. The findings show that H, L_b, t, and α are the most influential parameters for predicting rock displacement in tunnels and underground mines. In contrast, the contribution of b in rock displacement is tiny, and W_c did not relate to the rock displacement in tunnels and underground mines.
High speed and reconfigurable optronic neural network with digital nonlinear activation
2021, Optik
Citation Excerpt :
Compared to the opto-electrical hybrid neural network relied on digital neural network [30,31], our architecture makes full use of the parallel processing ability of the light, the whole system has lower spatial complexity and is reconfigurable. As shown in Fig. 1(a), the neural network realizes its function through the cross connection of neurons and by means of the back-propagation algorithm [32]. The core of its work includes matrix multiplication, activation and objective function.
With its unique parallel processing capability, optical neural network has shown low-power consumption in image recognition and speech processing. At present, the manufacturing technology of programmable photonic chip is not mature, and the realization of optical neural network in free-space is still a hot spot of intelligent optical computing. In this article, based on Mixed National Institute of Standards and Technology database (MNIST) datasets and 4 f system, three-layer optical neural networks are constructed, whose recognition accuracy can reach 93.66%. Our network is programmable, high speed, reconfigurable and is better than the existing free-space optical neural network in terms of spatial complexity.
Exploring the relation between production factors, ore grades, and life of mine for forecasting mining capital cost through a novel cascade forward neural network-based salp swarm optimization model
2021, Resources Policy
This paper aims at exploring the relationship between production factors, ore grades, and life of mine for forecasting mining capital cost (MCC) for open pit mining projects. Accordingly, the relationship between annual mine and mill production (MineAP, MillAP), stripping ratio (SR), reserve mean grade (RMG), the life of mine (LOM), and MCC of 80 open pit mining projects were investigated and thoroughly evaluated. The dataset was then divided into two sections, with 56 observations used to develop the forecast models. The remaining 24 observations were used to test the accuracy of the developed models. Subsequently, the cascade feedforward neural network (CFNN) was developed to forecast MCC based on the influential parameters. In order to improve the accuracy of the CFNN model, the salp swarm optimization (SalpSO) algorithm was applied to train the CFNN model and optimize the weights of the model, called the SalpSO-CFNN model. The benchmark models which were developed in the previous studies, such as support vector machine (SVM), classification and regression tree (CART), and multiple layers perceptron (MLP) neural network, were also developed in this study to compare with the proposed SalpSO-CFNN model in terms of MCC forecast. The results revealed that production factors, ore grades, and LOM are closely related to MCC, and they are statistically significant. The forecast results also indicated that the proposed novel SalpSO-CFNN model provided a good accuracy with a mean absolute error (MAE) of 179.567, root-mean-squared error (RMSE) of 248.401, and determination coefficient (R²) of 0.980. This result is higher by 18% compared with the CART model and 2–6% compared with the remaining forecast models. A sensitivity analysis also indicated that MineAP, MillAP are the most influential parameters on the forecast of MCC, and they should be specially taken into account when forecasting MCC of open pit mining projects.
Towards a mathematical framework to inform neural network modelling via polynomial regression
2021, Neural Networks
Citation Excerpt :
However, neural networks present several problems. Choosing the hyperparameters of NNs still depends mostly on an exploratory approach by trial and error, either for the learning algorithm parameters (Bengio, 2012) or for their structure topology like the needed number of layers, the number of hidden units per layer or their connections (Hirose et al., 1991; Ma & Khorasani, 2004; Weymaere & Martens, 1994), with genetic algorithms as an approach that has been explored to solve this (Leung et al., 2003). Another of their problems is that neural networks do not directly provide an estimate of the uncertainty produced in their predictions, which is of crucial importance in most of their applications, like flood predictions (Tiwari & Chatterjee, 2010), wind power forecasting (Wan et al., 2014) or molecular and atomic predictions (Musil et al., 2019).
Even when neural networks are widely used in a large number of applications, they are still considered as black boxes and present some difficulties for dimensioning or evaluating their prediction error. This has led to an increasing interest in the overlapping area between neural networks and more traditional statistical methods, which can help overcome those problems. In this article, a mathematical framework relating neural networks and polynomial regression is explored by building an explicit expression for the coefficients of a polynomial regression from the weights of a given neural network, using a Taylor expansion approach. This is achieved for single hidden layer neural networks in regression problems. The validity of the proposed method depends on different factors like the distribution of the synaptic potentials or the chosen activation function. The performance of this method is empirically tested via simulation of synthetic data generated from polynomials to train neural networks with different structures and hyperparameters, showing that almost identical predictions can be obtained when certain conditions are met. Lastly, when learning from polynomial generated data, the proposed method produces polynomials that approximate correctly the data locally.

View all citing articles on Scopus

View full text

Original contributionBack-propagation algorithm which varies the number of hidden units

Abstract

Learning representations by error propagation

Original contribution
Back-propagation algorithm which varies the number of hidden units