Skip to main content
SpringerLink
Log in

On Nonparametric Residual Variance Estimation

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this paper, the problem of residual variance estimation is examined. The problem is analyzed in a general setting which covers non-additive heteroscedastic noise under non-iid sampling. To address the estimation problem, we suggest a method based on nearest neighbor graphs and we discuss its convergence properties under the assumption of a Hölder continuous regression function. The universality of the estimator makes it an ideal tool in problems with only little prior knowledge available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Brown LD, Levine M (2006) Variance estimation in nonparametric regression via the difference sequence method. Technical report, Purdue University

  2. Devroye L, Wagner TJ (1979) Distribution-free probability inequalities for the deleted and holdout estimates. IEEE Trans Inf Theory 25(2): 202–207

    Article  MATH  MathSciNet  Google Scholar 

  3. Devroye L, Schäfer D, Györfi D, Walk H (2003) The estimation problem of minimum mean squared error. Stat Decis 21(1): 15–28

    Article  MATH  Google Scholar 

  4. Evans D (2002) Data-derived estimates of noise for unknown smooth models using near-neighbour asymptotics. Ph.D. thesis, Cardiff University

  5. Falconer KJ (1985) The geometry of fractal sets. Cambridge University Press, London

    MATH  Google Scholar 

  6. Hall P, Marron J (1990) On variance estimation in nonparametric regression. Biometrika 77(2)

  7. Hall P, Kay J, Titterington W (1990) Asymptotically optimal difference-based estimation of variance in nonparametric regression. Biometrika 77(3): 521–528

    Article  MathSciNet  Google Scholar 

  8. Jones AJ (2004) New tools in non-linear modelling and prediction. Comput Manage Sci 1(2): 109–149

    Article  MATH  Google Scholar 

  9. Kohler M (2006) Nonparametric regression with additional measurement errors in the dependent variable. J Stat Plann Infer 136(10): 3339–3361

    Article  MATH  MathSciNet  Google Scholar 

  10. Kulkarni SR, Posner SE (1995) Rates of convergence of nearest neighbor estimation under arbitrary sampling. IEEE Trans Infor Theory 41(4)

  11. Lendasse A, Ji Y, Reyhani N, Verleysen M (2005) LS-SVM hyperparameter selection with a nonparametric noise variance estimator. In: ICANN 2005, international conference on artifical neural networks. pp 625–630

  12. Liitiäinen E, Lendasse A, Corona F (2007) Non-parametric residual variance estimation in supervised learning. In: IWANN 2007, international work-conference on artifical neural networks. pp 63–71

  13. Massart P (2007) Concentration inequalities and model selection: Ecole d’Eté de Probabilités de Saint-Flour XXXIII-2003, vol 1896 of Lecture notes in mathematics / Ecole d’Eté Probabilités de Saint-Flour. Springer, Dordrecht

    Google Scholar 

  14. Müller U, Schik A, Wefelmeyer W (2003) Estimating the error variance in nonparametric regression by a covariate-matched U-statistic. Statistics 37(3): 179–188

    MATH  MathSciNet  Google Scholar 

  15. Pelckmans K, Brabanter JD, Suykens J, Moor BD (2004) The differogram: nonparametric noise variance estimation and its use for model selection. Neurocomputing 69(1–3): 100–122

    Google Scholar 

  16. Penrose MD (2008) Laws of large numbers in stochastic geometry with statistical applications. Bernoulli 13(4): 1124–1150

    Article  MathSciNet  Google Scholar 

  17. Rice J (1984) Bandwidth choice for nonparametric regression. Ann Stat 12(4): 1215–1230

    Article  MATH  MathSciNet  Google Scholar 

  18. Rudin W (1986) Real and complex analysis, Higher Mathematics Series. McGraw-Hill Science, NY

    Google Scholar 

  19. Ruppert D, Wand M, Holst U, Hossjer O (1997) Local polynomial variance-function estimation. Technometrics 39(3): 262–273

    Article  MATH  MathSciNet  Google Scholar 

  20. Shiryaev AN (1995) Probability. Springer, Dordrecht

    MATH  Google Scholar 

  21. Spokoiny V (2002) Variance estimation for high-dimensional regression models. J Multivariate Anal 10(4): 465–497

    MathSciNet  Google Scholar 

  22. Yao Q, Tong H (2000) Nonparametric estimation of ratios of noise to signal in nonparametric regression. Statistica Sinica 10(3): 751–770

    MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Helsinki University of Technology, P.O. Box 5400, 2015, Espoo, Finland

    Elia Liitiäinen, Francesco Corona & Amaury Lendasse

Authors
  1. Elia Liitiäinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesco Corona
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amaury Lendasse
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elia Liitiäinen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liitiäinen, E., Corona, F. & Lendasse, A. On Nonparametric Residual Variance Estimation. Neural Process Lett 28, 155–167 (2008). https://doi.org/10.1007/s11063-008-9087-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-008-9087-8

Keywords

Search

Navigation