Avoiding the Local Minima Problem in Backpropagation Algorithm with Modified Error Function

Weixing BI
Xugang WANG
Zheng TANG
Hiroki TAMURA

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E88-A    No.12    pp.3645-3653
Publication Date: 2005/12/01
Online ISSN: 
DOI: 10.1093/ietfec/e88-a.12.3645
Print ISSN: 0916-8508
Type of Manuscript: PAPER
Category: Neural Networks and Bioengineering
Keyword: 
backpropagation,  learning,  local minima,  modified error function,  

Full Text: PDF(547.6KB)>>
Buy this Article


Summary: 
One critical "drawback" of the backpropagation algorithm is the local minima problem. We have noted that the local minima problem in the backpropagation algorithm is usually caused by update disharmony between weights connected to the hidden layer and the output layer. To solve this kind of local minima problem, we propose a modified error function with two terms. By adding one term to the conventional error function, the modified error function can harmonize the update of weights connected to the hidden layer and those connected to the output layer. Thus, it can avoid the local minima problem caused by such disharmony. Simulations on some benchmark problems and a real classification task have been performed to test the validity of the modified error function.

open access publishing via