Skip to main content
SpringerLink
Log in

Manifold Learning in Regression Tasks

  • Conference paper
  • First Online:
Statistical Learning and Data Sciences (SLDS 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9047))

Included in the following conference series:

Abstract

The paper presents a new geometrically motivated method for non-linear regression based on Manifold learning technique. The regression problem is to construct a predictive function which estimates an unknown smooth mapping f from q-dimensional inputs to m-dimensional outputs based on a training data set consisting of given ‘input-output’ pairs. The unknown mapping f determines q-dimensional manifold M(f) consisting of all the ‘input-output’ vectors which is embedded in (q+m)-dimensional space and covered by a single chart; the training data set determines a sample from this manifold. Modern Manifold Learning methods allow constructing the certain estimator M* from the manifold-valued sample which accurately approximates the manifold. The proposed method called Manifold Learning Regression (MLR) finds the predictive function fMLR to ensure an equality M(fMLR) = M*. The MLR simultaneously estimates the m×q Jacobian matrix of the mapping f.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Vapnik, V.: Statistical Learning Theory. John Wiley, New-York (1998)

    MATH  Google Scholar 

  2. James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning with Applications in R. Springer Texts in Statistics, New-York

    Google Scholar 

  3. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer (2009)

    Google Scholar 

  4. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2007)

    Google Scholar 

  5. Deng, L., Yu, D.: Deep Learning: Methods and Applications. NOW Publishers, Boston (2014)

    Google Scholar 

  6. Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  7. Friedman, J.H.: Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics 29(5), 1189–1232 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  8. Rasmussen, C.E., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  9. Belyaev, M., Burnaev, E., Kapushev, Y.: Gaussian process regression for structured data sets. To appear in Proceedings of the SLDS 2015, London, England, UK (2015)

    Google Scholar 

  10. Burnaev E., Panov M.: Adaptive design of experiments based on gaussian processes. To appear in Proceedings of the SLDS 2015, London, England, UK (2015)

    Google Scholar 

  11. Loader, C.: Local Regression and Likelihood. Springer, New York (1999)

    MATH  Google Scholar 

  12. Vejdemo-Johansson, M.: Persistent homology and the structure of data. In: Topological Methods for Machine Learning, an ICML 2014 Workshop, Beijing, China, June 25 (2014). http://topology.cs.wisc.edu/MVJ1.pdf

  13. Carlsson, G.: Topology and Data. Bull. Amer. Math. Soc. 46, 255–308 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  14. Edelsbrunner, H., Harer, J.: Computational Topology: An Introduction. Amer. Mathematical Society (2010)

    Google Scholar 

  15. Cayton, L.: Algorithms for manifold learning. Univ of California at San Diego (UCSD), Technical Report CS2008-0923, pp. 541-555. Citeseer (2005)

    Google Scholar 

  16. Huo, X., Ni, X., Smith, A.K.: Survey of manifold-based learning methods. In: Liao, T.W., Triantaphyllou, E. (eds.) Recent Advances in Data Mining of Enterprise Data, pp. 691–745. World Scientific, Singapore (2007)

    Google Scholar 

  17. Ma, Y., Fu, Y. (eds.): Manifold Learning Theory and Applications. CRC Press, London (2011)

    Google Scholar 

  18. Bernstein, A.V., Kuleshov, A.P.: Tangent bundle manifold learning via grassmann&stiefel eigenmaps. In: arXiv:1212.6031v1 [cs.LG], pp. 1-25, December 2012

    Google Scholar 

  19. Bernstein, A.V., Kuleshov, A.P.: Manifold Learning: generalizing ability and tangent proximity. International Journal of Software and Informatics 7(3), 359–390 (2013)

    Google Scholar 

  20. Kuleshov, A., Bernstein, A.: Manifold learning in data mining tasks. In: Perner, P. (ed.) MLDM 2014. LNCS, vol. 8556, pp. 119–133. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  21. Kuleshov, A., Bernstein, A., Yanovich, Yu.: Asymptotically optimal method in Manifold estimation. In: Márkus, L., Prokaj, V. (eds.) Abstracts of the XXIX-th European Meeting of Statisticians, July 20-25, Budapest, p. 325 (2013)

    Google Scholar 

  22. Genovese, C.R., Perone-Pacifico, M., Verdinelli, I., Wasserman, L.: Minimax Manifold Estimation. Journal Machine Learning Research 13, 1263–1291 (2012)

    MATH  MathSciNet  Google Scholar 

  23. Kuleshov, A.P., Bernstein, A.V.: Cognitive Technologies in Adaptive Models of Complex Plants. Information Control Problems in Manufacturing 13(1), 1441–1452 (2009)

    Google Scholar 

  24. Bunte, K., Biehl, M., Hammer B.: Dimensionality reduction mappings. In: Proceedings of the IEEE Symposium on Computational Intelligence and Data Mining (CIDM 2011), pp. 349-356. IEEE, Paris (2011)

    Google Scholar 

  25. Lee, J.A.: Verleysen, M.: Quality assessment of dimensionality reduction: Rank-based criteria. Neurocomputing 72(7–9), 1431–1443 (2009)

    Article  Google Scholar 

  26. Saul, L.K., Roweis, S.T.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. Journal of Machine Learning Research 4, 119–155 (2003)

    MathSciNet  Google Scholar 

  27. Saul, L.K., Roweis, S.T.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)

    Article  Google Scholar 

  28. Zhang, Z., Zha, H.: Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment. SIAM Journal on Scientific Computing 26(1), 313–338 (2005)

    Article  MathSciNet  Google Scholar 

  29. Hamm, J., Lee, D.D.: Grassmann discriminant analysis: A unifying view on subspace-based learning. In: Proceedings of the 25th International Conference on Machine Learning (ICML 2008), pp. 376-83 (2008)

    Google Scholar 

  30. Tyagi, H., Vural, E., Frossard, P.: Tangent space estimation for smooth embeddings of riemannian manifold. In: arXiv:1208.1065v2 [stat.CO], pp. 1-35, May 17 (2013)

    Google Scholar 

  31. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15, 1373–1396 (2003)

    Article  MATH  Google Scholar 

  32. Bengio, Y., Monperrus, M.: Non-local manifold tangent learning. In: Advances in Neural Information Processing Systems, vol. 17, pp. 129-136. MIT Press, Cambridge (2005)

    Google Scholar 

  33. Dollár, P., Rabaud, V., Belongie, S.: Learning to traverse image manifolds. In: Advances in Neural Information Processing Systems, vol. 19, pp. 361-368. MIT Press, Cambridge (2007)

    Google Scholar 

  34. Xiong, Y., Chen, W., Apley, D., Ding, X.: A Nonstationary Covariance-Based Kriging Method for Metamodeling in Engineering Design. International Journal for Numerical Methods in Engineering 71(6), 733–756 (2007)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kharkevich Institute for Information Transmission Problems RAS, Moscow, Russia

    Alexander Bernstein, Alexander Kuleshov & Yury Yanovich

  2. National Research University Higher School of Economics, Moscow, Russia

    Alexander Bernstein, Alexander Kuleshov & Yury Yanovich

Authors
  1. Alexander Bernstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Kuleshov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yury Yanovich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yury Yanovich .

Editor information

Editors and Affiliations

  1. University of London, Egham, Surrey, United Kingdom

    Alexander Gammerman

  2. University of London, Egham, Surrey, United Kingdom

    Vladimir Vovk

  3. Frederick University, Nicosia, Cyprus

    Harris Papadopoulos

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Bernstein, A., Kuleshov, A., Yanovich, Y. (2015). Manifold Learning in Regression Tasks. In: Gammerman, A., Vovk, V., Papadopoulos, H. (eds) Statistical Learning and Data Sciences. SLDS 2015. Lecture Notes in Computer Science(), vol 9047. Springer, Cham. https://doi.org/10.1007/978-3-319-17091-6_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-17091-6_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-17090-9

  • Online ISBN: 978-3-319-17091-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation