Skip to main content

Experiments on regularizing MLP models with background knowledge

  • Part III: Learning: Theory and Algorithms
  • Conference paper
  • First Online:
Artificial Neural Networks — ICANN'97 (ICANN 1997)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1327))

Included in the following conference series:

Abstract

In this contribution we present results of using possibly inaccurate knowledge of model derivatives as part of the training data for a multilayer perceptron network (MLP). Even simple constraints offer significant improvements and the resulting models give better prediction performance than traditional data driven MLP models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Y. S. Abu-Mostafa, Hints and the VC dimension, Neural Computation, Vol. 5, No. 2, 1993, pp. 278–288.

    Google Scholar 

  2. C. Bishop, Curvature-Driven Smoothing: A Learning Algorithm for Feedforward Networks, IEEE Tr. on Neural Networks, Vol. 4, No. 5, Sep 1993, pp. 882–884.

    Google Scholar 

  3. C. Bishop, Regularization and Complexity Control in Feed-forward Networks, Proc. ICANN'95, Vol. 1., 1995, pp. 141–148.

    Google Scholar 

  4. C. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995.

    Google Scholar 

  5. S. Geman, E.Bienenstock and R.Doursat, Neural Networks and the Bias/Variance Dilemma, Neural Networks 4, 1992, pp. 1-58.

    Google Scholar 

  6. S. Haykin, Neural Networks, A Comprehensive Foundation, Macmillan, New York, NY, 1994.

    Google Scholar 

  7. G. Hinton, Connectionist Learning Procedures, Artificial Intelligence, 40, 1989, pp. 185–234.

    Google Scholar 

  8. J. Lampinen and O.Taipale, Optimization and Simulation of Quality Properties in Paper Machine with Neural Networks, Proc. of IEEE International Conference on Neural Networks, Orlando, FL, June 28–July 2, 1994, pp. 3812–3815.

    Google Scholar 

  9. J. Lampinen and A. Selonen, Multilayer Perceptron Training with Inaccurate Derivative Information, Proc. 1995 IEEE International Conference on Neural Networks ICNN'95, Perth, WA, Vol. 5, pp. 2811–2815, 1995.

    Google Scholar 

  10. Y. Le Cun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, L.D. Jackel Backpropagation applied to hand written zip code recognition, Neural Computation, Vol. 1, 1989, pp. 541–551.

    Google Scholar 

  11. M. Riedmiller and H. Braun, A direct adaptive method for faster backpropagation learning: The RPROP algorithm, in H. Ruspini, (Ed), Proc. IEEE Internation Conf. on Neural Networks, San Francisco, 1993, pp. 586–591.

    Google Scholar 

  12. A. Selonen, J. Lampinen, L. Ikonen, Using External Knowledge in Neural Network Models, in Proc. SPIE on Intelligent Robots and Computer Vision XV: Algorithms, Techniques, Active Vision, and Materials Handling, Vol. 2904, Boston, MA, 1996, pp. 239–249.

    Google Scholar 

  13. P. Y. Simard, B. Victorri, Y. LeCun and J. Denker, Tangent prop — a formalism for specifying selected invariances in an adaptive network, In Neural Information Processing Systems, Vol. 4, San Mateo, CA, 1992.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Wulfram Gerstner Alain Germond Martin Hasler Jean-Daniel Nicoud

Rights and permissions

Reprints and permissions

Copyright information

© 1997 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Selonen, A., Lampinen, J. (1997). Experiments on regularizing MLP models with background knowledge. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, JD. (eds) Artificial Neural Networks — ICANN'97. ICANN 1997. Lecture Notes in Computer Science, vol 1327. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0020182

Download citation

  • DOI: https://doi.org/10.1007/BFb0020182

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-63631-1

  • Online ISBN: 978-3-540-69620-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics