Abstract
The article considers a Gaussian model with the mean and the variance modeled flexibly as functions of the independent variables. The estimation is carried out using a Bayesian approach that allows the identification of significant variables in the variance function, as well as averaging over all possible models in both the mean and the variance functions. The computation is carried out by a simulation method that is carefully constructed to ensure that it converges quickly and produces iterates from the posterior distribution that have low correlation. Real and simulated examples demonstrate that the proposed method works well. The method in this paper is important because (a) it produces more realistic prediction intervals than nonparametric regression estimators that assume a constant variance; (b) variable selection identifies the variables in the variance function that are important; (c) variable selection and model averaging produce more efficient prediction intervals than those obtained by regular nonparametric regression.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aït-Sahalia Y. 1996a. Testing continuous-time models of the spot interest rate. The Review of Financial Studies 9: 385–426.
Aït-Sahalia Y. 1996b. Nonparametric pricing of interest rate derivative securities. Econometrica 64: 527–560.
Carroll R.J. (1982). Adapting for heteroscedasticity in linear models. Ann. Statist. 10: 1224–1233.
Carroll R.J. and Ruppert D. 1988. Transformation and Weighting in Regression. Chapman and Hall, London.
Chan K.C., Karolyi G.A., Longstaff F.A., and Sanders A.B. 1992. An empirical comparison of alternative models of the short-term interest rate. Journal of Finance 47: 1209–1227.
Cox J.C., Ingersoll J.E., and Ross S.A. 1985. A theory of the term structure of interest rates. Econometrica 53: 385–407.
Denison D.G.T., Mallick B.K., and Smith A.F.M. 1998. Automatic Bayesian curve fitting. Journal of the Royal Statistical Society B 60: 333–350.
Dette H., Munk A., and Wagner T. 1998. Estimating the variance in nonparametric regression?What is a reasonable choice? J. R. Statist. Soc. B 60: 751–764.
Eilers P.H.C. and Marx B.D. 1996. Flexible smoothing with B-splines and penalties. Statistical Science 11: 89–121.
Gamerman D. 1997. Sampling from the posterior distribution in generalized linear mixed models. Statistics and Computing 7(1): 57–68.
Hastie T. and Tibshirani R. 1990. Generalized Additive Models. Chapman and Hall, London.
Kass R.E. and Wasserman L. 1995. A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. Journal of the American Statistical Association 90: 928–934.
Kaufman L. and Rousseeuw P.J. 1990. Finding Groups in Data. John Wiley & Sons, New York.
Kohn R., Smith M., and Chan D. 2001. Nonparametric regression using linear combination of basis functions. Statistics and Computing 11: 301–310.
Marx B.D. and Eilers P.H.C. 1998. Direct generalized additive modelling with penalized likelihood. Computational Statistics and Data Analysis 28: 193–209.
Merton R.C. 1973. Theory of rational option pricing. Bell Journal of Economics and Management Science 4: 141–183.
Müller H.G. and Stadtmülleri U. 1987. Estimation of heteroscedasticity in regression analysis. Ann. Statist. 15: 610–625.
Powell M. 1987. Radial basis functions for multivariate interpolation: A review. In: Mason J. and Cox M. (Eds.), Algorithms of Approximation. Clarendon Press, Oxford, pp. 143–167.
Rendleman R. and Bartter B. 1980. The pricing of options on debt securities. Journal of Financial and Quantitative Analysis 15: 11–24.
Ruppert D. 2002. Selecting the number of knots for penalized splines. Journal of Computational and Graphical Statistics 11: 735–757.
Ruppert D. and Carroll R.J. 2000. Spatially-adaptive penalties for spline fitting. Australian and New Zealand Journal of Statistics 45: 205–223.
Ruppert D., Wand M.P., Holst U., and Hössjer O. 1997. Local polynomial variance-function estimation. Technometrics 39: 262–273.
Shively T., Kohn R., and Wood S. 1999. Variable selection and function estimation in additive nonparametric regression models using a data-based prior. (with discussion). Journal of the American Statistical Association 94: 777–806.
Silverman B.W. 1985. Some aspects of the spline smoothing approach to non-parametric regression curve fitting. J. R. Statist. Soc. B 47: 1–52.
Smith M. and Kohn R. 1996. Nonparametric regression using Bayesian variable selection. Journal of Econometrics 75: 317–344.
Smith M., Yau P., Shively T., and Kohn R. 2002. Estimating longterm trends in tropospheric ozone levels. International Statistical Review 70: 99–124.
Stanton R. 1997. A nonparametric model of term structure dynamics and the market price of interest rate risk. Journal of Finance 52: 1973–2002.
Struyf A., Hubert M., and Rousseeuw P. 1996. Clustering in an objectoriented environment. Journal of Statistical Software 1(4): 1–30.
Ullah A. 1985. Specification analysis of econometric models. Journal Quant. Econ. 2: 187–209.
Tierney L. 1994. Markov chains for exploring posterior distributions. The Annals of Statistics 22: 1703–1762.
Vasicek O.A. 1977. An equilibrium characterization of the term structure. Journal of Financial Economics 5: 177–188.
Wahba G. 1990. Spline Models for Observational Data. SIAM, Philadelphia.
Xiang D. and Wahba G. 1998. Approximate smoothing spline methods for large data sets in the binary case. In: Proceedings of the 1997 ASA Joint Statistical Meetings, Biometric Section, pp. 94–98.
Yau P., Kohn R., and Wood S. 2003. Bayesian variable selection and model averaging in high dimensional multinomial nonparametric regression. Journal of Computational and Graphical Statistics 11: 23–54.
Zellner A. 1986. On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In: Goel P.K. and Zellner A. (Eds.), Bayesian Inference and Decision Techniques-Essays in Honor of Bruno de Finetti. North-Holland, Amsterdam, pp. 233–243.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Yau, P., Kohn, R. Estimation and variable selection in nonparametric heteroscedastic regression. Statistics and Computing 13, 191–208 (2003). https://doi.org/10.1023/A:1024293931757
Issue Date:
DOI: https://doi.org/10.1023/A:1024293931757