Skip to main content
SpringerLink
Log in

Improving Bayesian Regularization of ANN via Pre-training with Early-Stopping

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

We propose a simple method that enhances the performance of Bayesian Regularization of Artificial Neural Network (ANN) through pre-training of initial network with the Early-Stopping algorithm. The proposed method is applied to the regularization of Feed-forward Neural Networks to regress three benchmark data series. Significant reduction in both the cross-validation error and the number of training over standard Bayesian Regularisation is achieved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. MacKay, D. J. C.: A practical Bayesian framework for backprop networks, In: J. E. Moody, S. J. Hanson and R. P. Lippmann (eds), Advances in Neural Information Processing Systems, pp.839–846, 1992.

  2. Vehtari, A. and Lampinen, J.: Bayesian neural networks for industrial applications. In: IEEE Midnight-Sun Workshop on Soft Computing Methods in Industrial Applications, Kuusamo, Finland, 1999.

  3. Edwards, P., Peacock, A., Renshaw, D., Hannah, J.and Murray, A.: Minimizing risk using prediction uncertainty in neural network estimation fusion and its application to paper-making, IEEE Transactions on Neural Networks, 13(3) (2002), 726–731.

    Google Scholar 

  4. Gencay, R.and Min, Q.: Pricing and hedging derivative securities with neural networks: Bayesian regularization, early stopping, and bagging, IEEE Transactions on Neural Networks, 12(4) (2001), 726–734.

    Google Scholar 

  5. Bishop, C.: Neural Network for Pattern Recognition, Oxford University Press Inc., 1995.

  6. Plate, T.: Controlling the hyperparameter search in MacKay 's Bayesian neural network Framework. In: G. B. Orr, K. R. Muller (eds), Neural Networks: Tricks of the Trade, Springer, pp. 93–112, 1998.

  7. MacKay, D. J. C.: Bayesian methods for neural networks-FAQ, http://wol.ra.phy.cam. ac.uk/mackay/Bayes_FAQ.html, 2000.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong

    Z. S. H. Chan, H. W. Ngan & A. B. Rad

Authors
  1. Z. S. H. Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H. W. Ngan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. B. Rad
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chan, Z.S.H., Ngan, H.W. & Rad, A.B. Improving Bayesian Regularization of ANN via Pre-training with Early-Stopping. Neural Processing Letters 18, 29–34 (2003). https://doi.org/10.1023/A:1026271406135

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1026271406135

Search

Navigation