Abstract
In this paper we study conditional quantile regression by learning algorithms generated from Tikhonov regularization schemes associated with pinball loss and varying Gaussian kernels. Our main goal is to provide convergence rates for the algorithm and illustrate differences between the conditional quantile regression and the least square regression. Applying varying Gaussian kernels improves the approximation ability of the algorithm. Bounds for the sample error are achieved by using a projection operator, a variance-expectation bound derived from a condition on conditional distributions and a tight bound for the covering numbers involving the Gaussian kernels.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68, 337–404 (1950)
Chen, D.R., Wu, Q., Ying, Y., Zhou, D.X.: Support vector machine soft margin classifiers: error analysis. J. Mach. Learn. Res. 5, 1143–1175 (2004)
Christman, A., Steinwart, I.: Consistency of kernel based quantile regression. Appl. Stoch. Models Bus. Ind. 24, 171–183 (2008)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press (2000)
Cucker, F., Zhou, D.X.: Learning Theory: An Approximation Theory Viewpoint. Cambridge University Press (2007)
De Vito, E., Caponnetto, A., Rosasco, L.: Model selection for regularized least-squares algorithm in learning theory. Found. Comput. Math. 5, 59–85 (2006)
De Vito, E., Rosasco, L., Caponnetto, A., Piana, M., Verri, A.: Some properties of regularized kernel methods. J. Mach. Learn. Res. 5, 1363–1390 (2004)
Evgeniou, T., Pontil, M., Poggio, T.: Regularization networks and support vector machines. Adv. Comput. Math. 13, 1–50 (2000)
Hardin, D., Tsamardinos, I., Aliferis, C.F.: A theoretical characterization of linear SVM-based feature selection. In: Proc. of the 21st Int. Conf. on Machine Learning. Banff, Canada (2004)
Smale, S., Zhou, D.X.: Estimating the approximation errors in learning theory. Anal. Appl. 1, 17–41 (2003)
Steinwart, I., Christmann, A.: How SVMs can estimate quantiles and the median. Adv. Neural Inf. Process. Syst. 20, 305–312 (2008)
Steinwart, I., Christman, A.: Estimating conditional quantiles with the help of the pinball loss. Bernoulli 17, 211–225 (2011)
Sun, H.W., Wu, Q.: Least square regression with indefinite kernels and coefficient regularization. Appl. Comput. Harmon. Anal. 30, 96–109 (2011)
Wahba, G.: Spline Models for Observational Data. SIAM (1990)
Wu, Q., Ying, Y., Zhou, D.X.: Multi-kernel regularized classifiers. J. Complex. 23, 108–134 (2007)
Xiang, D.H., Zhou, D.X.: Classification with Gaussians and convex loss. J. Mach. Learn. Res. 10, 1447–1468 (2009)
Yao, Y.: On complexity issue of online learning algorithms. IEEE Trans. Inf. Theory 56, 6470–6481 (2010)
Ying, Y.: Convergence analysis of online algorithms. Adv. Comput. Math. 27, 273–291 (2007)
Ying, Y., Zhou, D.X.: Learnability of Gaussians with flexible variances. J. Mach. Learn. Res. 8, 249–276 (2007)
Zhou, D.X.: The covering numbber in learning theory. J. Complex. 18, 739–767 (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Yuesheng Xu.
This work described in this paper is supported by NSF of China under Grant 11001247.
Rights and permissions
About this article
Cite this article
Xiang, DH. Conditional quantiles with varying Gaussians. Adv Comput Math 38, 723–735 (2013). https://doi.org/10.1007/s10444-011-9257-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10444-011-9257-5