Skip to main content
SpringerLink
Log in

The local minima of the error surface of the 2-2-1 XOR network

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

All local minima of the error surface of the 2-2-1 XOR network are described. A local minimum is defined as a point such that all points in a neighbourhood have an error value greater than or equal to the error value in that point. It is proved that the error surface of the two-layer XOR network with two hidden units has a number of regions with local minima. These regions of local minima occur for combinations of the weights from the inputs to the hidden nodes such that one or both hidden nodes are saturated for at least two patterns. However, boundary points of these regions of local minima are saddle points. It will be concluded that from each finite point in weight space a strictly decreasing path exists to a point with error zero. This also explains why experiments using higher numerical precision find less “local minima”.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. L.G.C. Hamey, XOR has no local minima: A case study in neural network error surface analysis, Neural Networks 11 (1998) 669-681.

    Article  Google Scholar 

  2. L.G.C. Hamey, Analysis of the error surface of the XOR network with two hidden nodes, Computing Report 95/167C, Department of Computing, Macquarie University, NSW 2109 Australia (1995). Accessible by FTP from ftp.mpce.mq.edu.au in pub/comp/techreports/950167.hamey.ps.

    Google Scholar 

  3. P.J.G. Lisboa and S.J. Perantonis, Complete solution of the local minima in the XOR problem, Network 2 (1991) 119-124.

    Article  MATH  MathSciNet  Google Scholar 

  4. L. Prechelt, A study of experimental evaluations of neural network learning algorithms: current research practice, Technical Report 19/94, Fakultät für Informatik, Universität Karlsruhe (1994).

  5. D.E. Rumelhart, J.L. McClelland and the PDP Research Group, Parallel Distributed Processing, Vol. 1 (MIT Press, Cambridge, MA, 1986).

    Google Scholar 

  6. I.G. Sprinkhuizen-Kuyper and E.J.W. Boers, The error surface of the simplest XOR network has only global minima, Neural Computation 8 (1996) 1301-1320.

    Google Scholar 

  7. I.G. Sprinkhuizen-Kuyper and E.J.W. Boers, The error surface of the 2-2-1 XOR network: the finite stationary points, Neural Networks 11 (1998) 683-690.

    Article  Google Scholar 

  8. I.G. Sprinkhuizen-Kuyper and E.J.W. Boers, The error surface of the 2-2-1 XOR network: stationary points with infinite weights, Technical Report 96-10, Department of Computer Science, Leiden University, The Netherlands (1996). Available via http://www.wi.leidenuniv.nl/TechRep/tr96-10.html.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Leiden University, Leiden, The Netherlands

    Ida G. Sprinkhuizen-Kuyper & Egbert J. W. Boers

Authors
  1. Ida G. Sprinkhuizen-Kuyper
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Egbert J. W. Boers
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sprinkhuizen-Kuyper, I.G., Boers, E.J.W. The local minima of the error surface of the 2-2-1 XOR network. Annals of Mathematics and Artificial Intelligence 25, 107–136 (1999). https://doi.org/10.1023/A:1018969803819

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018969803819

Keywords

Search

Navigation