Abstract
Recently many statistical methods for the design of neural networks were proposed. In this paper we present an integrated approach for building high performant classifiers for time series prediction. The classifier is based on a committee of neural networks, which are constructed to be as independent as possible. Each network is trained using a Bayesian learning rule. The training process is intertwined with the optimization of the network topology, especially the input structure. The mutual information of the input-output relation is exploited, to reduce the input dimension to the most possible limit. An evolutionary algorithm serves as search heuristic for the optimization process. The benefits of the approach are demonstrated on the development of a time series prediction system, that recently replaced its predecessor, which was trading online successfully since April 1996 at the stock market in Frankfurt, Germany. We conclude that by combining these efficient methods, one can increase considerably the performance of classifiers for time series prediction compared to conventional approaches.
Preview
Unable to display preview. Download preview PDF.
References
E.B. Baum and D. Haussler. What size net gives valid generalization ? Neural Computation, 1:151–160, 1989.
Christopher M. Bishop. Neural Networks for Pattern Recognition. Oxford Press, 1995.
B.V. Bonnlander and A.S. Weigend. Selecting Input Variables Using Mutual Information and Nonparametric Density Estimation. Proceedings of the ISANN '94, Taiwan, pages p.42–50, 1994.
H. Braun and T. Ragg. ENZO-User Manual and Implementation Guide, http://illwww.ira.uka.de. Technical Report 21/96, Universität Karlsruhe, 1996.
T.M. Cover and J.A. Thomas. Elements of Information Theory. Wiley Series in Telecommunications. John Wiley & Sons, 1991.
Terrence L. Fine. Theories of Probability. Academic Press, 1973.
K. Fukunaga. Introduction to Statistical Pattern Recognition. Acad. Press, 1990.
S. Gutjahr. Improving neural prediction systems using independent committees. In Proceedings of the NNCM '96. Springer, 1996.
D. MacKay. Bayesian methods for backpropagation networks. Models of Neural Networks III,, page Chapter 6, 1994.
D. MacKay. Hyperparameters: optimise or integrate out ? Maximum entropy and Bayesian methods, 1994.
M. Perrone and L. Cooper. When networks disagree: ensemble methods for hybrid neural networks. Artificial neural networks for speech and vision., pages 126–142, 1994.
T. Ragg and H. Braun. A Comparative Study of Neural Network Optimization Techniques. In Proceedings of the ICANNGA 97, Norwich, UK, 1997.
M. Riedmiller and H. Braun. A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm. In Proceedings of the ICNN, 1993.
C.E. Shannon. A mathematical theory of communication. Bell Sys. Tech. Journal, 27:379–423, 1948.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ragg, T., Gutjahr, S. (1997). Building high performant classifiers by Integrating bayesian learning, mutual Information and committee techniques — A case study in time series prediction —. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, JD. (eds) Artificial Neural Networks — ICANN'97. ICANN 1997. Lecture Notes in Computer Science, vol 1327. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0020287
Download citation
DOI: https://doi.org/10.1007/BFb0020287
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63631-1
Online ISBN: 978-3-540-69620-9
eBook Packages: Springer Book Archive