Abstract
Sparse grids, combined with gradient penalties provide an attractive tool for regularised least squares fitting. It has earlier been found that the combination technique, which builds a sparse grid function using a linear combination of approximations on partial grids, is here not as effective as it is in the case of elliptic partial differential equations. We argue that this is due to the irregular and random data distribution, as well as the proportion of the number of data to the grid resolution. These effects are investigated both in theory and experiments. As part of this investigation we also show how overfitting arises when the mesh size goes to zero. We conclude with a study of modified “optimal” combination coefficients who prevent the amplification of the sampling noise present while using the original combination coefficients.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Braess D (2001) Finite elements, 2nd edn. Cambridge University Press, Cambridge
Bramble JH, Xu J (1991) Some estimates for a weighted L 2 projection. Math Comput 56: 463–476
Brenner SC, Scott LR (2002) The mathematical theory of finite element methods. Texts in applied mathematics, 2nd edn, vol 15. Springer, New York
Bungartz H-J, Griebel M, Rüde U (1994) Extrapolation, combination, and sparse grid techniques for elliptic boundary value problems. Comput Methods Appl Mech Eng 116: 243–252
Garcke J (2004) Maschinelles Lernen durch Funktionsrekonstruktion mit verallgemeinerten dünnen Gittern. Doktorarbeit, Institut für Numerische Simulation, Universität Bonn
Garcke J (2006) Regression with the optimised combination technique. In: Cohen W, Moore A (eds) Proceedings of the 23rd ICML ’06. ACM Press, New York, NY, USA, pp 321–328
Garcke J (2008) An optimised sparse grid combination technique for eigenproblems. In: Proceedings of ICIAM 2007, PAMM, vol 7, pp 1022301–1022302
Garcke J, Griebel M (2002) Classification with sparse grids using simplicial basis functions. Intelligent data analysis 6:483–502 (shortened version appeared in KDD 2001, Proc. of the Seventh ACM SIGKDD, F. Provost and R. Srikant (eds), pp 87–96, ACM, 2001)
Garcke J, Griebel M, Thess M (2001) Data mining with sparse grids. Computing 67: 225–253
Gilbarg D, Trudinger NS (2001) Elliptic partial differential equations of second order. Classics in mathematics. Springer, Berlin
Griebel M, Schneider M, Zenger C (1992) A combination technique for the solution of sparse grid problems. In: Groen P, Beauwens R (eds) Iterative methods in linear algebra. IMACS, Elsevier, North Holland, pp 263–281
Hackbusch W (1992) Elliptic differential equations. Springer series in computational mathematics, vol 18. Springer, Berlin
Hegland M (2003) Additive sparse grid fitting. In: Proceedings of the fifth international conference on curves and surfaces, Saint-Malo, France 2002, pp 209–218. Nashboro Press
Hegland M, Garcke J, Challis V (2007) The combination technique and some generalisations. Linear Algebra Appl 420: 249–275
Natterer F (1977) Regularisierung schlecht gestellter Probleme durch Projektionsverfahren. Numer Math 28: 329–341
Schölkopf B, Smola A (2002) Learning with kernels. MIT Press, Cambridge
Tikhonov AN, Arsenin VA (1977) Solutions of ill-posed problems. W.H. Winston, Washington D.C.
Wahba G (1990) Spline models for observational data. Series in applied mathematics, vol 59. SIAM, Philadelphia
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Garcke, J., Hegland, M. Fitting multidimensional data using gradient penalties and the sparse grid combination technique. Computing 84, 1–25 (2009). https://doi.org/10.1007/s00607-009-0027-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00607-009-0027-x