Abstract
In this contribution we present results of using possibly inaccurate knowledge of model derivatives as part of the training data for a multilayer perceptron network (MLP). Even simple constraints offer significant improvements and the resulting models give better prediction performance than traditional data driven MLP models.
Preview
Unable to display preview. Download preview PDF.
References
Y. S. Abu-Mostafa, Hints and the VC dimension, Neural Computation, Vol. 5, No. 2, 1993, pp. 278–288.
C. Bishop, Curvature-Driven Smoothing: A Learning Algorithm for Feedforward Networks, IEEE Tr. on Neural Networks, Vol. 4, No. 5, Sep 1993, pp. 882–884.
C. Bishop, Regularization and Complexity Control in Feed-forward Networks, Proc. ICANN'95, Vol. 1., 1995, pp. 141–148.
C. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995.
S. Geman, E.Bienenstock and R.Doursat, Neural Networks and the Bias/Variance Dilemma, Neural Networks 4, 1992, pp. 1-58.
S. Haykin, Neural Networks, A Comprehensive Foundation, Macmillan, New York, NY, 1994.
G. Hinton, Connectionist Learning Procedures, Artificial Intelligence, 40, 1989, pp. 185–234.
J. Lampinen and O.Taipale, Optimization and Simulation of Quality Properties in Paper Machine with Neural Networks, Proc. of IEEE International Conference on Neural Networks, Orlando, FL, June 28–July 2, 1994, pp. 3812–3815.
J. Lampinen and A. Selonen, Multilayer Perceptron Training with Inaccurate Derivative Information, Proc. 1995 IEEE International Conference on Neural Networks ICNN'95, Perth, WA, Vol. 5, pp. 2811–2815, 1995.
Y. Le Cun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, L.D. Jackel Backpropagation applied to hand written zip code recognition, Neural Computation, Vol. 1, 1989, pp. 541–551.
M. Riedmiller and H. Braun, A direct adaptive method for faster backpropagation learning: The RPROP algorithm, in H. Ruspini, (Ed), Proc. IEEE Internation Conf. on Neural Networks, San Francisco, 1993, pp. 586–591.
A. Selonen, J. Lampinen, L. Ikonen, Using External Knowledge in Neural Network Models, in Proc. SPIE on Intelligent Robots and Computer Vision XV: Algorithms, Techniques, Active Vision, and Materials Handling, Vol. 2904, Boston, MA, 1996, pp. 239–249.
P. Y. Simard, B. Victorri, Y. LeCun and J. Denker, Tangent prop — a formalism for specifying selected invariances in an adaptive network, In Neural Information Processing Systems, Vol. 4, San Mateo, CA, 1992.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Selonen, A., Lampinen, J. (1997). Experiments on regularizing MLP models with background knowledge. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, JD. (eds) Artificial Neural Networks — ICANN'97. ICANN 1997. Lecture Notes in Computer Science, vol 1327. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0020182
Download citation
DOI: https://doi.org/10.1007/BFb0020182
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63631-1
Online ISBN: 978-3-540-69620-9
eBook Packages: Springer Book Archive