Abstract
A Bayesian approach to multivariate adaptive regression spline (MARS) fitting (Friedman, 1991) is proposed. This takes the form of a probability distribution over the space of possible MARS models which is explored using reversible jump Markov chain Monte Carlo methods (Green, 1995). The generated sample of MARS models produced is shown to have good predictive power when averaged and allows easy interpretation of the relative importance of predictors to the overall fit.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Becker, R., Chambers, J. M. and Wilks, A. (1988) The New S Language, Wadsworth, Belmont, California.
Breiman, L. and Friedman, J. H. (1985) Estimating optimal transformations for multiple regression and correlation (with discussion). Journal of the American Statistical Association, 80, 580–619.
Breiman, L., Friedman, J. H., Olshen, R. and Stone, C. J. (1984) Classification and Regression Trees, Wadsworth, Belmont, California.
Bruntz, S. M., Cleveland, W. S., Kleiner, B. and Warner, J. L. (1974) The dependence of ambient ozone on solar radiation, temperature and mixing height, in Symposium on atmospheric diffusion and air pollution. American Meteorological Society, Boston, pp. 125–8.
Chipman, H., George, E. I. and McCulloch, R. E. (1998) Bayesian CART model search (with discussion). Journal of the American Statistical Association 93, 935–960.
Cleveland, W. S. and Devlin, S. J. (1988) Locally-weighted regression: an approach to regression analysis by local fitting. Journal of the American Statistical Association, 83, 597–610.
Craven, P. and Wahba, G. (1979) Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of cross-validation. Numerische Mathematik, 31, 317–403.
Denison, D. G. T. (1997) Simulation based Bayesian nonparametric regression methods. Unpublished Ph.D. Thesis. Imperial College, London.
Denison, D. G. T., Mallick, B. K. and Smith, A. F. M. (1998a) A Bayesian CART algorithm. Biometrika, 85, 363–377.
Denison, D. G. T., Mallick, B. K. and Smith, A. F. M. (1998b) Automatic Bayesian curve fitting. Journal of the Royal Statistical Society, Series B, 60, 333–350.
Friedman, J. H. (1991) Multivariate adaptive regression splines (with discussion). The Annals of Statistics, 19, 1–141.
Friedman, J. H., Grosse, E. and Stuetzle, W. (1983) Multidimensional additive spline approximation. SIAM Journal of Scientific and Statistical Computing, 291–301.
Friedman, J. H. and Stuetzle, W. (1981) Projection pursuit regression. Journal of the American Statistical Association, 76, 817–823.
Gelfand, A. E. and Smith, A. F. M. (1990) Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.
George, E. I. and McCulloch, R. E. (1993) Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88, 881–889.
Green, P. J. (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82, 771–32.
Hastie, T. J. and Tibshirani, R. J. (1990) Generalized Additive Models, Chapman & Hall, London.
Hastings, W. K. (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97–109.
Holmes, C. C. and Mallick, B. K. (1997) Bayesian wavelet networks for non-parametric regression. Technical report. Imperial College, London.
Hwang, J-N, Lay, S-R, Maechler, M., Martin, D. and Schimert, J. (1994) Regression modeling in back-propagation and projection pursuit learning. IEEE Transactions on Neural Networks, 5, 342–53.
Kass, R. E. and Raftery, A. E. (1995) Bayes factors. Journal of the American Statistical Association, 90, 773–795.
Mallick, B. K., Denison, D. G. T. and Smith, A. F. M. (1997) Bayesian survival analysis using a MARS model. Technical report. Imperial College, London.
Mallick, B. K., Denison, D. G. T. and Smith, A. F. M. (1998) Semiparametric generalized linear models: Bayesian approaches, in Generalized Linear Models: A Bayesian Perspective, Dey, D. K., Ghosh, S. K. and Mallick, B. K. (eds) Marcel-Dekker (to appear).
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953) Equations of state calculations by fast computing machines. Journal of Chemical Physics, 21, 1087–1091.
Morgan, J. N. and Sonquist, J. A. (1963) Problems in the analysis of survey data and a proposal. Journal of the American Statistical Association, 58, 415–434.
O'Hagan, A. (1994) Kendall's Advanced Theory of Statistics, Volume 2B, Edward Arnold, London.
Richardson, S. and Green, P. J. (1997) Bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Society (Ser. B), 59, 731–792.
Roosen, C. B. and Hastie, T. J. (1994) Automatic smoothing spline projection pursuit. Journal of Computational and Graphical Statistics, 3, 235–48.
Tierney, L. (1994) Markov chains for exploring posterior distributions (with discussion). The Annals of Statistics, 22, 1701–1762.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
DENISON, D.G.T., MALLICK, B.K. & SMITH, A.F.M. Bayesian MARS. Statistics and Computing 8, 337–346 (1998). https://doi.org/10.1023/A:1008824606259
Issue Date:
DOI: https://doi.org/10.1023/A:1008824606259