Bandwidth Selection for Nadaraya-Watson Kernel Estimator Using Cross-Validation Based on Different Penalty Functions

Abstract

The traditional cross-validation usually selects an over-smoothing bandwidth for kernel regression. The penalty function based cross-validation (e.g., generalized cross-validation ($\mathrm{{CV}}_{\mathrm{{GCV}}}$), the Shibata’s model selector ($\mathrm{{CV}}_{\mathrm{{S}}}$), the Akaike’s information criterion ($\mathrm{{CV}}_{\mathrm{{AIC}}}$) and the Akaike’s finite prediction error ($\mathrm{{CV}}_{\mathrm{{FPE}}}$)) are introduced to relieve the problem of selecting over-smoothing bandwidth parameter by the traditional cross-validation for kernel regression problems. In this paper, we investigate the influence of these four different penalty functions on the cross-validation based bandwidth selection in the framework of a typical kernel regression method, i.e., the Nadaraya-Watson kernel estimator (NWKE). Firstly, we discuss the mathematical properties of these four penalty functions. Then, experiments are given to compare the performance of aforementioned cross-validation methods. Finally, we give guidelines for the selection of different penalty functions in practical applications.