Skip to main content
SpringerLink
Log in

Adaptive Metric Kernel Regression

  • Published:
Journal of VLSI signal processing systems for signal, image and video technology Aims and scope Submit manuscript

Abstract

Kernel smoothing is a widely used non-parametric pattern recognition technique. By nature, it suffers from the curse of dimensionality and is usually difficult to apply to high input dimensions. In this contribution, we propose an algorithm that adapts the input metric used in multivariate regression by minimising a cross-validation estimate of the generalisation error. This allows to automatically adjust the importance of different dimensions. The improvement in terms of modelling performance is illustrated on a variable selection task where the adaptive metric kernel clearly outperforms the standard approach. Finally, we benchmark the method using the DELVE environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. W. Härdle, Applied Nonparametric Regression, Cambridge University Press, 1990. Econometric Society Monographs, vol. 19.

  2. B.D. Ripley, Pattern Recognition and Neural Networks, Cambridge University Press, 1996.

  3. H. Schiøler and U. Hartmann, “Mapping Neural Networks Derived from the Parzen Window Estimator,” Neural Networks, vol. 5, no.6, 1992, pp. 903–909.

    Article  Google Scholar 

  4. D.F. Specht, “A General Regression Neural Network,” IEEE Transactions on Neural Networks, vol. 2, no.6, 1991, pp. 568–576.

    Article  Google Scholar 

  5. R.E. Bellman, Adaptive Control Processes: A Guided Tour, New Jersey: Princeton University Press, 1961.

    MATH  Google Scholar 

  6. E.A. Nadaraya, “On Estimating Regression,” Theory Probab. Appl., vol. 10, 1964, pp. 186–190.

    Article  Google Scholar 

  7. G.S. Watson, “Smooth Regression Analysis,” Sankhyā Ser. A, vol. 26, 1964, pp. 101–116.

    MATH  Google Scholar 

  8. M.P. Wand and M.C. Jones, Kernel Smoothing, London: Chapman & Hall, 1995. Monographs on Statistics and Applied Probability, vol. 60.

    Book  MATH  Google Scholar 

  9. V.N. Vapnik, The Nature of Statistical Learning Theory, Springer, 1995.

  10. C. Goutte, “Statistical Learning and Regularisation in Regression,” Ph.D. Thesis 1997/033, Université Paris 6, Paris, July 1997. http://eivind.imm.dtu.dk/staff/goutte/PUBLIS/thesis.html.

  11. M. Stone, “Cross-Validatory Choice and Assessment of Statistical Predictions,” Journal of the Royal Statistical Society B, vol. 36, 1974, pp. 111–147, with discussion.

    MATH  Google Scholar 

  12. C. Goutte and J. Larsen, “Adaptive Regularization of Neural Networks Using Conjugate gradient,” in Proceedings of ICASSP'98, 1998, vol. 2, IEEE, pp. 1205–1208. ftp://eivind.imm.dtu.dk/dist/1998/goutte.icassp98.ps.Z.

    Google Scholar 

  13. R. Fletcher, Practical Methods of Optimization, Wiley, 1987.

  14. J.H. Friedman, “Multivariate Adaptive Regression Splines,” The Annals of Statistics, vol. 19, no.1, 1991, pp. 1–141.

    Article  MathSciNet  MATH  Google Scholar 

  15. C. Goutte and J. Larsen, “Optimal Cross-Validation Split Ratio: Experimental Investigation,” in ICANN'98, Berlin, 1998, L. Niklasson, M. Bodén, and T. Ziemke (Eds.), vol. 2, Springer Verlag, pp. 681–686. ftp://eivind.imm.dtu.dk/dist/1998/goutte.icann98.html.

  16. C.E. Rasmussen, R.M. Neal, G.E. Hinton, D. van Camp, M. Revow, Z. Ghahramani, R. Kustra, and R. Tibshirani, The DELVE Manual, version 1.1 edition, Dec. 1996. http://www.cs.utoronto.ca/~delve/.

  17. C.E. Rasmussen, “Evaluation of Gaussian Processes and Other Methods for Non-Linear Regression,” PhD Thesis, Department of Computer Science, University of Toronto, 1996. http://bayes.imm.dtu.dk/.

  18. D. Harrison and D.L. Rubinfeld, “Hedonic Prices and the Demand for Clean Air,” J. Environ. Economics and Management, vol. 5, 1978, pp. 81–102.

    Article  MATH  Google Scholar 

  19. R.M. Neal, Bayesian Learning for Neural Networks, New York: Springer, 1996. Lecture Notes in Statistics, vol. 118.

    Book  MATH  Google Scholar 

  20. D. Lowe, “Similarity Metric Learning for a Variable-Kernel Classifier,” Neural Computation, vol. 7, no.1, 1995, pp. 72–85.

    Article  Google Scholar 

  21. C. Goutte and L.K. Hansen, “Regularization with a Pruning Prior,” Neural Networks, vol. 10, no.6, 1997, pp. 1053–1059.

    Article  Google Scholar 

  22. C.K.I. Williams, “Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond,” Technical Report NCRG/97/012, Neural Computing Research Group, Aston University, UK, 1997. http://neural-sever.aston.ac.uk/Papers/postscript/NCRG 97 012.ps.Z.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Modelling, Technical University of Denmark, 2800, Lyngby, Denmark

    Cyril Goutte & Jan Larsen

Authors
  1. Cyril Goutte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Larsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Goutte, C., Larsen, J. Adaptive Metric Kernel Regression. The Journal of VLSI Signal Processing-Systems for Signal, Image, and Video Technology 26, 155–167 (2000). https://doi.org/10.1023/A:1008159803952

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008159803952

Keywords

Search

Navigation