Abstract
This paper considers the estimation of monotone nonlinear regression functions based on Support Vector Machines (SVMs), Least Squares SVMs (LS-SVMs) and other kernel machines. It illustrates how to employ the primal-dual optimization framework characterizing LS-SVMs in order to derive a globally optimal one-stage estimator for monotone regression. As a practical application, this letter considers the smooth estimation of the cumulative distribution functions (cdf), which leads to a kernel regressor that incorporates a Kolmogorov–Smirnoff discrepancy measure, a Tikhonov based regularization scheme and a monotonicity constraint.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
V. N. Vapnik (1998) Statistical Learning Theory Wiley New York
B. Schölkopf A. Smola (2002) Learning with Kernels MIT Press Cambridge, MA
Poggio, T. and Girosi, F.: Networks for approximation and learning, Proceedings of the IEEE, 78, (september 1990), pp. 1481–1497.
Hastie, T., Tibshirani, R. and Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer-Verlag, 2001.
E. Mammen J. S. Marron B. A. Turlach M. P. Wand (2001) ArticleTitleA general projection framework for constrained smoothing Statistical Science 16 IssueID3 232–248 Occurrence Handle10.1214/ss/1009213727 Occurrence HandleMR1874153
Suykens, J. A. K., Van Gestel, T. De Brabanter, J. De Moor, B. and Vandewalle, J.: Least Squares Support Vector Machines, World Scientific, 2002.
Suykens, J. A. K., Horvath, G. Basu, S. Micchelli, C. Vandewalle, J. (eds.), Advances in Learning Theory: Methods, Models and Applications, NATO Science Series III: Computer & Systems Sciences, p. 190, IOS Press Amsterdam, 2003.
Pelckmans, K. Goethals, I. De Brabanter, J. Suykens, J. A. K. De Moor, B.: Componentwise least squares support vector machines, Internal Report 04-75, ESAT-SISTA, K.U.Leuven (Leuven, Belgium), 2004.
Chebyshev, P. L.: Sur les questions de minima qui se rattachent la representation approximative des fonctions, Mé moires Academie des Science Petersburg. 7, 199-291, 1859. Oeuvres de P. L. Tchebychef, 1, 273–378, Chelsea, New York, 1961 (1859).
Scott, D. W.: Multivariate Density Estimation, Theory, Practice and Visualization,Wiley series inb probability and mathematical statistics, 1992.
D. E. Gaylord C. K. Ramirez (1991) ArticleTitleMonotone regression splines for smoothed bootstrapping. Computational Statistics Quarterly 6 IssueID2 85–97
Silverman, B. W.: Density Estimation for Statistics and Data Analysis, Monographs on Statistics and Applied Probability, Chapman & Hall, 1986, p. 26.
P. Billingsley (1986) Probability and Measure Wiley New York 26–97
Devroye, L.: Non-Uniform Random Variate Generation, Springer-Verlag, 1986.
W.J. Conover (1971) Practical Nonparametric Statistics Wiley New York
C. De Boor B. Schwartz (1977) ArticleTitlePiecewise monotone interpolation J. Approximation Theory 21 411–416 Occurrence Handle10.1016/0021-9045(77)90011-9
J.O. Ramsay (1988) ArticleTitleMonotone regression splines in action Statistical Science 3 425–461
Vapnik, V. and Mukherjee, S.: Support vector method for multivariate density estimation. In: S. A. Solla T. K. Leen and K.-R. Müller (eds.), In Advances in Neural Information Processing Systems, Vol. 12, pp. 659–665, 1999.
A. N. Tikhonov V. Y. Arsenin (1977) Solution of Ill-Posed Problems Winston Washington, DC
Boyd, S. and Vandenberghe, L.: Convex Optimization, Cambridge University Press, 2004.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pelckmans, K., Espinoza, M., De Brabanter, J. et al. Primal-Dual Monotone Kernel Regression. Neural Process Lett 22, 171–182 (2005). https://doi.org/10.1007/s11063-005-5264-1
Issue Date:
DOI: https://doi.org/10.1007/s11063-005-5264-1