Abstract
In the context of semi-functional partial linear regression model, we study the problem of error density estimation. The unknown error density is approximated by a mixture of Gaussian densities with means being the individual residuals, and variance a constant parameter. This mixture error density has a form of a kernel density estimator of residuals, where the regression function, consisting of parametric and nonparametric components, is estimated by the ordinary least squares and functional Nadaraya–Watson estimators. The estimation accuracy of the ordinary least squares and functional Nadaraya–Watson estimators jointly depends on the same bandwidth parameter. A Bayesian approach is proposed to simultaneously estimate the bandwidths in the kernel-form error density and in the regression function. Under the kernel-form error density, we derive a kernel likelihood and posterior for the bandwidth parameters. For estimating the regression function and error density, a series of simulation studies show that the Bayesian approach yields better accuracy than the benchmark functional cross validation. Illustrated by a spectroscopy data set, we found that the Bayesian approach gives better point forecast accuracy of the regression function than the functional cross validation, and it is capable of producing prediction intervals nonparametrically.
Similar content being viewed by others
References
Aneiros-Pérez G, Cao R, Vilar-Fernández JM, Muñoz-San-Roque A (2011) Functional prediction for the residual demand in electricity spot markets. In: Ferraty F (ed) Recent advances in functional data analysis and related topics. Physica-Verlag, Berlin
Aneiros-Pérez G, Vieu P (2006) Semi-functional partial linear regression. Stat Probab Lett 76(11): 1102–1110
Aneiros-Pérez G, Vieu P (2008) Nonparametric time series prediction: a semi-functional partial linear modeling. J Multivar Anal 99(5):834–857
Aneiros-Pérez G, Vieu P (2011) Automatic estimation procedure in partial linear model with functional data. Stat Papers 52(4):751–771
Anglin PM, Gencay R (1996) Semiparametric estimation of a hedonic price function. J Appl Econom 11(6):633–648
Ansley CF, Wecker WE (1983) Extensions and examples of the signal extraction approach to regression. In: Zellner A (ed) Applied time series analysis of economic data. Bureau of the Census, Washington, pp 181–192
Barrientos-Marin J, Ferraty F, Vieu P (2010) Locally modelled regression and functional data. J Nonparametr Stat 22(5):617–632
Benhenni K, Ferraty F, Rachdi M, Vieu P (2007) Local smoothing regression with functional data. Comput Stat 22(3):353–369
Berlinet A, Elamine A, Mas A (2011) Local linear regression for functional data. Ann Inst Stat Math 63(5):1047–1075
Boj E, Delicado P, Fortiana J (2010) Distance-based local linear regression for functional predictors. Comput Stat Data Anal 54(2):429–437
Bowman AW (1984) An alternative method of cross-validation for the smoothing of density estimates. Biometrika 71(2):353–360
Burba F, Ferraty F, Vieu P (2009) \(k\)-nearest neighbour method in functional nonparametric regression. J Nonparametr Stat 21(4):453–469
Engle R, Granger C, Rice J, Weiss A (1986) Semiparametric estimates of the relation between weather and electricity sales. J Am Stat Assoc 81(394):310–320
Eubank RL, Whitney P (1989) Convergence rates for estimation in certain partially linear models. J Stat Plan Inf 23(1):33–43
Ferraty F, Van Keilegom I, Vieu P (2010) On the validity of the bootstrap in non-parametric functional regression. Scand J Stat 37(2):286–306
Ferraty F, Vieu P (2002) The functional nonparametric model and application to spectrometric data. Comput Stat 17(4):545–564
Ferraty F, Vieu P (2006) Nonparametric functional data analysis: theory and practice. Springer, New York
Ferraty F, Vieu P (2009) Additive prediction and boosting for functional data. Comput Stat Data Anal 53(4):1400–1413
Gabrys R, Horváth L, Kokoszka P (2010) Tests for error correlation in the functional linear model. J Am Stat Assoc 105(491):1113–1125
Garthwaite PH, Fan Y, Sisson SA (2010) Adaptive optimal scaling of Metropolis-Hastings algorithms using the Robbins-Monro process, Working paper, University of New South Wales. http://arxiv.org/pdf/1006.3690v1.pdf
Geweke J (1992) Evaluating the accuracy of sampling-based approaches to calculating posterior moments. In: Bernardo JM, Berger J (eds) Bayesian statistics. Clarendon Press, Oxford, pp 169–193
Geweke J (1999) Using simulation methods for Bayesian econometric models: inference, development, and communication. Econom Rev 18(1):1–73
Geweke J (2010) Complete and incomplete econometric models. Princeton University Press, Princeton
Gilks WR, Richardson S, Spiegelhalter DJ (1996) Markov chain Monte Carlo in practice. Chapman & Hall, London
Goutis C (1998) Second-derivative functional regression with applications to near infra-red spectroscopy. J R Stat Soc Ser B 60(1):103–114
Hall P (1987) On Kullback-Leibler loss and density estimation. Ann Stat 15(4):1491–1519
Härdle W, Liang H, Gao J (2000) Partially linear models. Physica-Verlag, New York
Heckman N (1986) Spline smoothing in a partly linear model. J R Stat Soc Ser B 48(2):244–248
Heidelberger P, Welch PD (1983) Simulation run length control in the presence of an initial transient. Oper Res 31(6):1109–1144
Horváth L, Kokoszka P (2012) Inference for functional data with applications. Springer, New York
Jaki T, West RW (2008) Maximum kernel likelihood estimation. J Comput Graph Stat 17(4):976–993
Jaki T, West W (2011) Symmetric maximum kernel likelihood estimation. J Stat Comput Simul 81(2): 193–206
Kim S, Shephard N, Chib S (1998) Stochastic volatility: likelihood inference and comparison with arch models. Rev Econ Stud 65(3):361–393
Marron JS, Wand MP (1992) Exact mean integrated squared error. Ann Stat 20(2):712–736
McLachlan G, Peel D (2000) Finite mixture models. Wiley, New York
Meyer R, Yu J (2000) BUGS for a Bayesian analysis of stochastic volatility models. Econom J 3(2):198–215
Plummer M, Best N, Cowles K, Vines K (2006) Coda: convergence diagnosis and output analysis for mcmc. R News 6(1):7–11
R Core Team (2013) R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/
Rachdi M, Vieu P (2007) Nonparametric regression for functional data: automatic smoothing parameter selection. J Stat Plan Inf 137(9):2784–2801
Ramsay JO, Silverman BW (2005) Functional data analysis, 2nd edn. Springer, New York
Rice J (1986) Convergence rates for partially splined models. Stat Probab Lett 4(4):203–208
Robbins H, Monro S (1951) A stochastic approximation method. Ann Math Stat 22(3):400–407
Robert CP, Casella G (2010) Introducing Monte Carlo methods with R. Springer, New York
Roberts GO, Rosenthal JS (2009) Examples of adaptive MCMC. J Comput Graph Stat 18(2):349–367
Robinson P (1988) Root-N-consistent semiparametric regression. Econometrica 56(4):931–954
Samb R (2011) Nonparametric estimation of the density of regression errors. C R Acad Sci Paris Ser I 349(23–24), 1281–1285
Schmalensee R, Stoker TM (1999) Household gasoline demand in the united states. Econometrica 67(3):645–662
Shang HL (2013a) Bayesian bandwidth estimation for a nonparametric functional regression model with unknown error density. Comput Stat Data Anal 67:185–198
Shang HL (2013b) Functional time series approach for forecasting very short-term electricity demand. J Appl Stat 40(1):152–168
Speckman P (1988) Kernel smoothing in partial linear models. J R Stat Soc Ser B 50(3):413–436
Tse YK, Zhang X, Yu J (2004) Estimation of hyperbolic diffusion using the Markov chain Monte Carlo method. Quant Financ 4(2):158–169
Yao F, Müller H-G (2010) Functional quadratic regression. Biometrika 97(1):49–64
Zhang X, Brooks RD, King ML (2009) A Bayesian approach to bandwidth selection for multivariate kernel regression with an application to state-price density estimation. J Econom 153(1):21–32
Zhang X, King ML (2011) Bayesian semiparametric GARCH models, Working paper, Monash University. http://www.buseco.monash.edu.au/ebs/pubs/wpapers/2011/wp24-11.pdf
Acknowledgments
The author thanks Professors Rob Hyndman and Donald Poskitt for introducing him to functional data analysis, and Professors Xibin Zhang and Maxwell King for introducing him to Bayesian bandwidth estimation. The author would like to acknowledge Monash Sun Grid for its excellent parallel computing facility. Special thanks go to an associate editor and a referee for their insightful comments and suggestions, which led to a much improved manuscript.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Shang, H.L. Bayesian bandwidth estimation for a semi-functional partial linear regression model with unknown error density. Comput Stat 29, 829–848 (2014). https://doi.org/10.1007/s00180-013-0463-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00180-013-0463-0