Abstract
The local linear kernel estimator (LLKE) is a typical kernel-type regression method which is a non-parametric method to estimate the conditional expectation of a random variable and the non-linear mapping from input to output. There are three commonly used LLKEs, i.e., the Nadaraya-Watson kernel estimator, the Priestley-Chao kernel estimator and the Gasser-Müller kernel estimator. Existing studies show that the performance of LLKE mainly depends on the selection of an important parameter, i.e. bandwidth \(h\), when a special kernel function is employed. However, there is no comparative research conducted to study the effectiveness of different kernel functions. In this paper, we compare the performance of three aforementioned LLKEs based on 6 different kernel functions (i.e., Gaussian, uniform, Epanechnikov, biweight, triweight and cosine kernels) on their estimation error measured by the mean squared error (i.e., \(mse\)) and stability of method measured by the standard deviation of \(mse\) (i.e., \(std\)). Finally, we give guidelines for the selection of LLKE method and corresponding kernel function in practical applications.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Watson, G.S.: Smooth Regression Analysis. Sankhyā: The Indian Journal of Statistics. Series A 26(4), 359–372 (1964)
Clevelanda, W.S., Devlin, S.J.: Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting. Journal of the American Statistical Association 83(403), 596–610 (1988)
Chatterjee, S., Hadi, A.S.: Regression Analysis by Example. John Wiley & Sons (2006)
Montgomery, D.C., Peck, E.A., Vining, G.G.: Introduction to Linear Regression Analysis. John Wiley & Sons (2012)
Seber, G.A.F., Lee, A.J.: Linear Regression Analysis. John Wiley & Sons (2012)
Liu, Z.X., Liu, J.H.: Chaotic Time Series Multi-step Direct Prediction with Partial Least Squares Regression. Journal of Systems Engineering and Electronics 18(3), 611–615 (2007)
Murtagh, F., Spagat, M., Restrepo, J.A.: Ultrametric Wavelet Regression of Multivariate Time Series: Application to Colombian Conflict Analysis. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 41(2), 254–263 (2011)
Zhang, X.R., Hu, L.Y., Wang, Z.S.: Multiple Kernel Support Vector Regression for Economic Forecasting. In: Proceedings of ICMSE 2010, pp. 129–134 (2010)
Zhang, S., Wu, Y.: The Application of Regression Analysis in Correlation Research of Economic Growth in Jilin Province and Regional Income Distribution Gap. In: Proceedings of WKDD 2009, pp. 500–503 (2009)
Fan, J.Q., Heckman, N.E., Wand, M.P.: Local Polynomial Kernel Regression for Generalized Linear Models and Quasi-Likelihood Functions. Journal of the American Statistical Association 90(429), 141–150 (1995)
Watson, G.S.: Smooth Regression Analysis. Sankhy: The Indian Journal of Statistics, Series A (1961–2002) 26(4), 359–372 (1964)
Priestley, M.B., Chao, M.T.: Non-Parametric Function Fitting. Journal of the Royal Statistical Society Series B (Methodological) 34(3), 385–392 (1972)
Gasser, T., Müller, H.G.: Kernel estimation of regression functions. Smoothing Techniques for Curve Estimation, Lecture Notes in Mathematics 757, 23–68 (1979)
Parzen, E.: On Estimation of a Probability Density Function and Mode. Annals of Mathematical Statistics 33(3), 1065–1076 (1962)
Wang, X.Z., He, Y.L., Wang, D.D.: Non-Naive Bayesian Classifiers for Classification Problems with Continuous Attributes. IEEE Transactions on Cybernetics (2013),doi: 10.1109/TCYB.2013.2245891
Scott, D.W.: Multivariate Density Estimation: Theory, Practice, and Visualization. John Wiley and Sons, Inc. (1992)
Wand, M.P., Jones, M.C.: Kernel Smoothing. Chapman and Hall (1995)
Liu, J.N.K., He, Y.L., Wang, X.Z., Hu, Y.X.: A Comparative Study among Different Kernel Functions in Flexible Naive Bayesian Classification. In: Proceedings of ICMLC 2011, pp. 638–643 (2011)
Liu, J.N.K., He, Y.L., Wang, X.Z.: Improving Kernel Incapability by Equivalent Probability in Flexible Naive Bayesian. In: Proceedings of FUZZ-IEEE 2012, pp. 1–8 (2012)
Sun, Z.L., Au, K.F., Choi, T.M.M.: A Neuro-Fuzzy Inference System Through Integration of Fuzzy Logic and Extreme Learning Machines. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 37(5), 1321–1331 (2007)
Härdle, W.: Applied Nonparametric Regression. Cambridge University Press (1994)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, J., He, Y. (2014). Comparative Analysis of Density Estimation Based Kernel Regression. In: Wang, X., Pedrycz, W., Chan, P., He, Q. (eds) Machine Learning and Cybernetics. ICMLC 2014. Communications in Computer and Information Science, vol 481. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-45652-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-662-45652-1_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-45651-4
Online ISBN: 978-3-662-45652-1
eBook Packages: Computer ScienceComputer Science (R0)