Skip to main content

Building high performant classifiers by Integrating bayesian learning, mutual Information and committee techniques — A case study in time series prediction —

  • Part VII:Prediction, Forecasting, and Monitoring
  • Conference paper
  • First Online:
Artificial Neural Networks — ICANN'97 (ICANN 1997)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1327))

Included in the following conference series:

Abstract

Recently many statistical methods for the design of neural networks were proposed. In this paper we present an integrated approach for building high performant classifiers for time series prediction. The classifier is based on a committee of neural networks, which are constructed to be as independent as possible. Each network is trained using a Bayesian learning rule. The training process is intertwined with the optimization of the network topology, especially the input structure. The mutual information of the input-output relation is exploited, to reduce the input dimension to the most possible limit. An evolutionary algorithm serves as search heuristic for the optimization process. The benefits of the approach are demonstrated on the development of a time series prediction system, that recently replaced its predecessor, which was trading online successfully since April 1996 at the stock market in Frankfurt, Germany. We conclude that by combining these efficient methods, one can increase considerably the performance of classifiers for time series prediction compared to conventional approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. E.B. Baum and D. Haussler. What size net gives valid generalization ? Neural Computation, 1:151–160, 1989.

    Google Scholar 

  2. Christopher M. Bishop. Neural Networks for Pattern Recognition. Oxford Press, 1995.

    Google Scholar 

  3. B.V. Bonnlander and A.S. Weigend. Selecting Input Variables Using Mutual Information and Nonparametric Density Estimation. Proceedings of the ISANN '94, Taiwan, pages p.42–50, 1994.

    Google Scholar 

  4. H. Braun and T. Ragg. ENZO-User Manual and Implementation Guide, http://illwww.ira.uka.de. Technical Report 21/96, Universität Karlsruhe, 1996.

    Google Scholar 

  5. T.M. Cover and J.A. Thomas. Elements of Information Theory. Wiley Series in Telecommunications. John Wiley & Sons, 1991.

    Google Scholar 

  6. Terrence L. Fine. Theories of Probability. Academic Press, 1973.

    Google Scholar 

  7. K. Fukunaga. Introduction to Statistical Pattern Recognition. Acad. Press, 1990.

    Google Scholar 

  8. S. Gutjahr. Improving neural prediction systems using independent committees. In Proceedings of the NNCM '96. Springer, 1996.

    Google Scholar 

  9. D. MacKay. Bayesian methods for backpropagation networks. Models of Neural Networks III,, page Chapter 6, 1994.

    Google Scholar 

  10. D. MacKay. Hyperparameters: optimise or integrate out ? Maximum entropy and Bayesian methods, 1994.

    Google Scholar 

  11. M. Perrone and L. Cooper. When networks disagree: ensemble methods for hybrid neural networks. Artificial neural networks for speech and vision., pages 126–142, 1994.

    Google Scholar 

  12. T. Ragg and H. Braun. A Comparative Study of Neural Network Optimization Techniques. In Proceedings of the ICANNGA 97, Norwich, UK, 1997.

    Google Scholar 

  13. M. Riedmiller and H. Braun. A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm. In Proceedings of the ICNN, 1993.

    Google Scholar 

  14. C.E. Shannon. A mathematical theory of communication. Bell Sys. Tech. Journal, 27:379–423, 1948.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Wulfram Gerstner Alain Germond Martin Hasler Jean-Daniel Nicoud

Rights and permissions

Reprints and permissions

Copyright information

© 1997 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Ragg, T., Gutjahr, S. (1997). Building high performant classifiers by Integrating bayesian learning, mutual Information and committee techniques — A case study in time series prediction —. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, JD. (eds) Artificial Neural Networks — ICANN'97. ICANN 1997. Lecture Notes in Computer Science, vol 1327. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0020287

Download citation

  • DOI: https://doi.org/10.1007/BFb0020287

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-63631-1

  • Online ISBN: 978-3-540-69620-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics