Skip to main content

Advertisement

Log in

Genetic algorithms for the selection of smoothing parameters in additive models

  • Published:
Computational Statistics Aims and scope Submit manuscript

Summary

Additive models of the type y=f1(x1)+...+fp(xp)+ε where fj, j=1,..,p, have unspecified functional form, are flexible statistical regression models which can be used to characterize nonlinear regression effects. One way of fitting additive models is the expansion in B-splines combined with penalization which prevents overfitting. The performance of this penalized B-spline (called P-spline) approach strongly depends on the choice of the amount of smoothing used for components fj. In particular for higher dimensional settings this is a computationaly demanding task. In this paper we treat the problem of choosing the smoothing parameters for P-splines by genetic algorithms. In several simulation studies this approach is compared to various alternative methods of fitting additive models. In particular functions with different spatial variability are considered and the effect of constant respectively local adaptive smoothing parameters is evaluated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9

Similar content being viewed by others

References

  • Baker, J.: 1987, Reducing bias and inefficiency in the selection algorithm, in J. Greffenstette (ed.), Proceedings of the Second International Conference on Genetic Algorithms, Lawrence Erlbaum Associates, Hillsdale, NJ, pp. 14–21.

    Google Scholar 

  • Biller, C. and Fahrmeir, L: 2002, Bayesian varying-coefficient models using adaptive regression splines, Statistical Modelling 11, 1–17.

    MATH  Google Scholar 

  • Buja, A., Hastie, T. J. and Tibshirani, R. J.: 1989, Linear smoothers and additive models (with discussion), Annals of Statistics 17, 453–555.

    Article  MathSciNet  Google Scholar 

  • Chapelle, O. and Vapnik, V.: 1999, Model selection for support vector machines, in S. A. Solla, T. K. Leen and K. R. Müller (eds), Advances in Neural Information Processing Systems 12, MIT Press, Cambridge, MA.

    Google Scholar 

  • de Boor, C.: 1978, A Practical Guide to Splines, Springer, New York, Heidelberg, Berlin.

    Book  Google Scholar 

  • deJong, K. A.: 1975, An Arzalysis of the Behaviour of a Class of Genetic Adaptive Systems, Ph.D. Thesis, Department of Computer and Communication Sciences, University of Michigan, An Arbor.

    Google Scholar 

  • Dierckx, P.: 1995, Curve and Surface Fitting with Splines, Clarendon Press, Oxford.

    MATH  Google Scholar 

  • Eilers, P. H. C. and Marx, B. D.: 1996, Flexible smoothing with b-splines and penalties, Stat. Science 11(2), 89–121.

    Article  MathSciNet  Google Scholar 

  • Friedman, J.: 1991, Multivariate adaptive regression splines (with discussion), Annals of Statistics 19(1), 1–141.

    Article  MathSciNet  Google Scholar 

  • Goldberg, D. E.: 1989 Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Reading, MA.

    MATH  Google Scholar 

  • Gu, C. and Wahba, G.: 1991, Minimizing gcv/gml scores with multiple smoothing parameters via the newton methods, SIAM Journal of Scientific and Statistical Computing 12, 383–398.

    Article  MathSciNet  Google Scholar 

  • Hastie, T. J. and Tibshirani, R. J.: 1990, Generalized Additive Models, Chapman and Hall, London.

    MATH  Google Scholar 

  • Hastie, T. J., Tibshirani, R. J. and Friedman, J.: 2001, The Elements of Statistical Learning, Springer, New York.

    Book  Google Scholar 

  • Herrera, F., Lozano, M. and Verdegay, J. L.: 1998, Tackling real-coded genetic algorithms: Operators and tools for behavioural analysis, Artificial Intelligence Review 12 (4), 265–319.

    Article  Google Scholar 

  • Holland, J.: 1975, Adaption in neural and artificial systems, University of Michigan Press, Ann Arbor.

    Google Scholar 

  • Hurvich, C. and Simonoff, J.: 1998, Smoothing parameter selection in nonparametric regression using an improved akaike information criterion, Journal of the Royal Statistical Society B 60(2), 271–293.

    Article  MathSciNet  Google Scholar 

  • Krause, R.: 2004, Genetic Algorithms as Tool for Statistical Analysis of High-Dimensional Data Structures, Logos Verlag, Berlin.

    MATH  Google Scholar 

  • Lang, S. and Brezger, A.: 2004, Bayesian p-splines, Journal of Computational and Graphical Statistics 13, 183–212.

    Article  MathSciNet  Google Scholar 

  • Mammen, E., Linton, O. and Nielsen, J. P.: 1999, The existence and asymptotic properties of a backfitting projection algorithm under weak conditions, Annals of Statistics 27, 1443–1490.

    MathSciNet  MATH  Google Scholar 

  • Michalewicz, Z.: 1996, Genetic Algorithms+Data Structures=Evolution Programs, Springer, Berlin, Heidelberg.

    Book  Google Scholar 

  • Parise, H., Wand, M. P., Ruppert, D. and Ryan, L.: 2001, Incorporation of historical controls using semiparametric mixed models, Applied Statistics 50(1), 31–42.

    MATH  Google Scholar 

  • Rawlings, J. O., Pantula, S. G. and Dickey, D. A.: 1998, Applied Regression Analysis, Springer, New York.

    Book  Google Scholar 

  • Ruppert, D. and Carroll, R.: 2000, Spatially-adaptive penalties for spline fitting, Australian and New Zealand Journal of Statistics 42(2), 205–223.

    Article  Google Scholar 

  • Tipping, M. E.: 2001, Sparse bayesian learning and the relevance vector machine, Journal of Machine Learning Research 1, 211–244.

    MathSciNet  MATH  Google Scholar 

  • Tjostheim, D. and Auestad, B.: 1994, Nonparametric identification of nonlinear time series: projections, Journal of the American Statistical Association 89, 1398–1409.

    MathSciNet  MATH  Google Scholar 

  • Wood, S.: 2000, Modelling and smoothing parameter estimation with multiple quadratic penalties, Journal of the Royal Statistical Society B 62(2), 413–428.

    Article  MathSciNet  Google Scholar 

  • Wood, S.: 2001, mgcv: Gams and generalized ridge regression for R, R News 1 (2), 20–25.

    Google Scholar 

Download references

Acknowledgement

We thank Stefan Lang, Thomas Kneib and David Rummel for assistance and the supply of some simulation results we have used in this study.

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Krause, R., Tutz, G. Genetic algorithms for the selection of smoothing parameters in additive models. Computational Statistics 21, 9–31 (2006). https://doi.org/10.1007/s00180-006-0248-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00180-006-0248-9

Keywords

Navigation