Skip to main content
SpringerLink
Log in

Statistical Learning on Manifold-Valued Data

  • Conference paper
  • First Online:
Book cover Machine Learning and Data Mining in Pattern Recognition (MLDM 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9729))

  • 3054 Accesses

Abstract

Regression on manifolds problem is to estimate an unknown smooth function f that maps p-dimensional manifold-valued inputs, whose values lie on unknown Input manifold M of lower dimensionality q < p embedded in an ambient high-dimensional input space Rp, to m-dimensional outputs from training sample consisting of given ‘input-output’ pairs. We consider this problem in which Jacobian Jf(X) of function f and Input manifold M should be also estimated. The paper presents a new geometrically motivated method for estimating a triple (f(X), Jf(X), M) from given sample. The proposed solution is based on solving a Tangent bundle manifold learning problem for specific unknown Regression manifold embedded in input-output space Rp+m and consisting of input-output pairs (X, f(X)), X ∈ M.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Vapnik, V.: Statistical Learning Theory. John Wiley, New York (1998)

    MATH  Google Scholar 

  2. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, New York (2009)

    Book  MATH  Google Scholar 

  3. James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning with Applications in R. Springer Texts in Statistics. Springer, New York (2013)

    Book  MATH  Google Scholar 

  4. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2007)

    MATH  Google Scholar 

  5. Friedman, J.H.: Greedy Function Approximation: A Gradient Boosting Machine. Ann. Stat. 29(5), 1189–1232 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  6. Rasmussen, C.E., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  7. Loader, C.: Local Regression and Likelihood. Springer, New York (1999)

    MATH  Google Scholar 

  8. Wang, G.: Gary Shan S.: Review of metamodeling techniques in support of engineering design optimization. J. Mech. Des. 129(3), 370–381 (2007)

    Article  Google Scholar 

  9. Forrester, A.I.J., Sobester, A., Keane, A.J.: Engineering Design via Surrogate Modelling: A Practical Guide. Wiley, New York (2008)

    Book  Google Scholar 

  10. Kuleshov, A.P., Bernstein, A.V.: Cognitive technologies in adaptive models of complex plants. Inf. Control Probl. Manuf. 13(1), 1441–1452 (2009)

    Google Scholar 

  11. Stone, C.J.: Optimal global rates of convergence for nonparametric regression. Ann. Stat. 10, 1040–1053 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  12. Seung, H.S., Lee, D.D.: The Manifold Ways of Perception. Science 290(5500), 2268–2269 (2000)

    Article  Google Scholar 

  13. Rajaram, D., Pant, R.S.: An improved methodology for airfoil shape optimization using surrogate based design optimization. In: Rodrigues, H., et al. (eds.) Engineering Optimization IV, pp. 147–152. CRC Press, Taylor & Francis Group, London (2015)

    Google Scholar 

  14. Bernstein, A., Kuleshov, A., Sviridenko, Y., Vyshinsky, V.: Fast aerodynamic model for design technology. In: Proceedings of West-East High Speed Flow Field Conference (WEHSFF-2007), Moscow, Russia (2007). http://wehsff.imamod.ru/pages/s7.htm

  15. Zhu, F., Qin, N., Burnaev, E.V., Bernstein, A.V., Chernova, S.S.: Comparison of three geometric parameterization methods and their effect on aerodynamic optimization. In: Poloni, C. (ed.) Eurogen 2011, Optimization and Control with Applications to Industrial and Societal Problems International Conference on Proceedings - Evolutionary and Deterministic Methods for Design, pp. 758–772. Sira, Capua (2011)

    Google Scholar 

  16. Pelletier, B.: Nonparametric regression estimation on closed Riemannian manifolds. J. Nonparametric Stat. 18(1), 57–67 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  17. Loubes, J.-M., Pelletier, B.: A kernel-based classifier on a Riemannian manifold. Statistics and Decisions 26(1), 35–51 (2008). Verlag, Oldenbourg

    Article  MathSciNet  MATH  Google Scholar 

  18. Bickel, P., Li, B.: Local polynomial regression on unknown manifolds. In: Complex Datasets and Inverse Problems: Tomography, Networks and Beyond . IMS Lecture notes – Monograph Series, vol. 54, pp. 177–186 (2007)

    Google Scholar 

  19. Aswani, A., Bickel, P., Tomlin, C.: Regression on manifolds: Estimation of the exterior derivative. Ann. Stat. 39(1), 48–81 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  20. Cheng, M.-Y., Wu, H.-T.: Local Linear Regression on Manifolds and its Geometric Interpretation. J. Am. Stat. Assoc. 108(504), 1421–1434 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Yang, Y., Dunson, D.B.: Bayesian manifold regression. In: arXiv:1305.0167v2 [math.ST], pp. 1–40, June 16, 2014

  22. Guhaniyogi, R., Dunson, D.B.: Compressed gaussian process. In: arXiv:1406.1916v1 [stat.ML], pp. 1–29, June 7, 2014

  23. Fletcher, P.T.: Geodesic regression on Riemannian manifolds. In: Proceedings of International Workshop on Mathematical Foundations of Computational Anatomy (MFCA), pp. 75–86 (2011)

    Google Scholar 

  24. Hinkle, J., Muralidharan, P., Fletcher, P., Joshi, S.: Polynomial regression on riemannian manifolds. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 1–14. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  25. Steinke, F., Hein, M., Schölkopf, B.: Nonparametric regression between general Riemannian manifolds. SIAM J. Imaging Sci. 3(3), 527–563 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  26. Levina, E., Bickel, P.J.: Maximum likelihood estimation of intrinsic dimension. In: Saul, L., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 17, pp. 777–784. MIT Press, Cambridge (2005)

    Google Scholar 

  27. Fan, M., Qiao, H., Zhang, B.: Intrinsic dimension estimation of manifolds by incising balls. Pattern Recogn. 42, 780–787 (2009)

    Article  MATH  Google Scholar 

  28. Fan, M., Gu, N., Qiao, H., Zhang, B.: Intrinsic dimension estimation of data by principal component analysis. In: arXiv:1002.2050v1 [cs.CV], pp. 1–8, February 10, 2010

  29. Rozza, A., Lombardi, G., Rosa, M., Casiraghi, E., Campadelli, P.: IDEA: intrinsic dimension estimation algorithm. In: Maino, G., Foresti, G.L. (eds.) ICIAP 2011, Part I. LNCS, vol. 6978, pp. 433–442. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  30. Ma, Y., Fu, Y. (eds.): Manifold Learning Theory and Applications. CRC Press, London (2011)

    Google Scholar 

  31. Bernstein, A.V., Kuleshov, A.P.: Tangent bundle manifold learning via Grassmann&Stiefel eigenmaps. In: arXiv:1212.6031v1 [cs.LG], pp. 1–25, December 2012

  32. Bernstein, A.V., Kuleshov, A.P.: Manifold Learning: generalizing ability and tangent proximity. Int. J. Softw. Inf. 7(3), 359–390 (2013)

    Google Scholar 

  33. Bernstein, A., Kuleshov, A., Yanovich, Y.: Manifold learning in regression tasks. In: Gammerman, A., Vovk, V., Papadopoulos, H. (eds.) SLDS 2015. LNCS, vol. 9047, pp. 414–423. Springer, Heidelberg (2015)

    Chapter  Google Scholar 

  34. Niyogi, P., Smale, S., Weinberger, S.: Finding the homology of submanifolds with high confidence from random samples. Discrete Comput. Geom. 39, 419–441 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  35. Bernstein, A.V., Kuleshov, A.P.: Data-based manifold reconstruction via tangent bundle manifold learning. In: ICML-2014, Topological Methods for Machine Learning Workshop, Beijing, June 25, 2014. http://topology.cs.wisc.edu/KuleshovBernstein.pdf

  36. Kuleshov, A., Bernstein, A.: Manifold learning in data mining tasks. In: Perner, P. (ed.) MLDM 2014. LNCS, vol. 8556, pp. 119–133. Springer, Heidelberg (2014)

    Google Scholar 

  37. Kuleshov, A., Bernstein, A., Yanovich, Yu.: Asymptotically optimal method in manifold estimation. In: Márkus, L., Prokaj, V. (eds.) Abstracts of the XXIX-th European Meeting of Statisticians, July 20–25, 2013, Budapest, p. 325 (2013)

    Google Scholar 

  38. Genovese, C.R., Perone-Pacifico, M., Verdinelli, I., Wasserman, L.: Minimax Manifold Estimation. J. Mach. Learn. Res. 13, 1263–1291 (2012)

    MathSciNet  MATH  Google Scholar 

  39. Golub, G.H., Van Loan, C.F.: Matrix Computation, 3rd edn. Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  40. Wasserman, L.: All of Nonparametric Statistics. Springer Texts in Statistics, Berlin (2007)

    Google Scholar 

  41. Jollie, T.: Principal Component Analysis. Springer, New York (2002)

    Google Scholar 

  42. Singer, A., Wu, H.-T.: Vector Diffusion Maps and the Connection Laplacian. Commun. Pure Appl. Math. 65(8), 1067–1144 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  43. Tyagi, H., Vural, E., Frossard, P.: Tangent space estimation for smooth embeddings of Riemannian manifold. In: arXiv:1208.1065v2 [stat.CO], pp. 1–35, May 17, 2013

  44. Hamm, J., Lee, D.D.: Grassmann discriminant analysis: a unifying view on subspace-based learning. In: Proceedings of the 25th International Conference on Machine Learning (ICML 2008), pp. 376–383 (2008)

    Google Scholar 

  45. Wolf, L., Shashua, A.: Learning over sets using kernel principal angles. J. Mach. Learn. Res. 4, 913–931 (2003)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Skolkovo Institute of Science and Technology, Moscow, Russia

    Alexander Kuleshov

  2. Institute for Systems Analysis, FRC CSC RAS, Moscow, Russia

    Alexander Bernstein

  3. Kharkevich Institute for Information Transmission Problems RAS, Moscow, Russia

    Alexander Bernstein

Authors
  1. Alexander Kuleshov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Bernstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Bernstein .

Editor information

Editors and Affiliations

  1. IBaI, Inst of Comp Vision and applied Comp Sci, Leipzig, Sachsen, Germany

    Petra Perner

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Kuleshov, A., Bernstein, A. (2016). Statistical Learning on Manifold-Valued Data. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2016. Lecture Notes in Computer Science(), vol 9729. Springer, Cham. https://doi.org/10.1007/978-3-319-41920-6_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41920-6_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41919-0

  • Online ISBN: 978-3-319-41920-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation