Elsevier

Neurocomputing

Volume 16, Issue 2, 31 July 1997, Pages 117-126
Neurocomputing

Paper
Training a feed-forward network by feeding gradients forward rather than by back-propagation of errors

https://doi.org/10.1016/S0925-2312(97)00020-9Get rights and content

Abstract

This paper demonstrates how a multi-layer feed-forward network may be trained, using the method of gradient descent, by feeding gradients forward rather than by feeding errors backwards as is usual in the case of back-propagation. The gradient of steepest descent requires that the gradient of the output of the network with respect to each connection matrix be calculated and that the output of the final layer be calculated. The work in this paper shows how the gradients of the final output are determined by feeding the gradients of the intermediate outputs forward at the same time that the outputs of the intermediate layers are fed forward in order to determine the output of the final layer. This method turns out to be equivalent to back propagation for a 2-layer network but is much more readily extended to several layers. The method makes obvious a great potential for concurrency and the algorithm may be directly implemented using an array processing language. It may be used, without modification, for a network with arbitrary number of layers. The logic complexity of the algorithm is independent of the number of layers in the network.

References (5)

  • K.E. Iverson

    Elementary Functions: An Algorithmic Treatment

    (1966)
  • B.D. Ripley

    Neural networks and related methods for classification

    J. Royal Statist. Soc.

    (1994)
There are more references available in the full text version of this article.

Cited by (15)

  • Determination of the effects of operating conditions on the output power of the inverter and the power quality using an artificial neural network

    2019, Engineering Science and Technology, an International Journal
    Citation Excerpt :

    Hence, the ANN has one input layer with 6 input data, one hidden layer with 10 neurons and one output layer with 3 output data as demonstrated in Fig. 4. In ANN step of this study, feed forward back propagation (FFBP) ANN and Levenberg–Marquardt (LM) algorithm which is intensively employed in many studies were chosen due to the reliable, simple and rapid properties although several training algorithms are existing in literature [11,29–31]. In addition, Learngdm adaptation learning function, mean square error (MSE) performance function, and Tansig transfer function were preferred in this ANN simulation.

View all citing articles on Scopus
View full text