Skip to main content
SpringerLink
Log in

Embedding new data points for manifold learning via coordinate propagation

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

In recent years, a series of manifold learning algorithms have been proposed for nonlinear dimensionality reduction. Most of them can run in a batch mode for a set of given data points, but lack a mechanism to deal with new data points. Here we propose an extension approach, i.e., mapping new data points into the previously learned manifold. The core idea of our approach is to propagate the known coordinates to each of the new data points. We first formulate this task as a quadratic programming, and then develop an iterative algorithm for coordinate propagation. Tangent space projection and smooth splines are used to yield an initial coordinate for each new data point, according to their local geometrical relations. Experimental results and applications to camera direction estimation and face pose estimation illustrate the validity of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Baldi P, Hornik K (1989) Neural networks and principal component analysis: learning from examples without local minima. Neural Netw 2(1): 53–58

    Article  Google Scholar 

  2. Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6): 1373–1396

    Article  MATH  Google Scholar 

  3. Bengio Y, Paiement JF, Vincent P (2004) Out-of-sample extensions for lle, isomap, mds, eigenmaps and spectral clustering. In: Advances in neural information processing systems, p 16

  4. Bishop CM, Svensn M, Williams CKI (1998) Gtm: the generative topographic mapping. Neural Comput 10(1): 215–234

    Article  Google Scholar 

  5. Boyd S, Vandenberghe L (2004) Convex optimization. Cambriage University Press, Cambridge

    MATH  Google Scholar 

  6. Brand M (2003) Charting a manifold. In: Advances in neural information processing systems, vol 15. MIT Press, Cambridge, pp 961–968

  7. Brun A, Park HJ, Kuntsson H, Westin CF (2003) Coloring of dt-mri fiber traces using laplacian eigenmaps. In: Proceedings of international conference on computer aided systems theory, Las Palmas, Spain, pp 518–529

  8. Cai C, He XF, Han JW (2007) Efficient kernel discriminant analysis via spectral regression, Technical Report UIUCDCS-R-2007-2888, Department of Computer Science, University of Illinois at Urbana-Champaign

  9. Carreira-Perpiñán MÁ, Lu ZD (2007) The laplacian eigenmaps latent variable models. In: International conference on artificial intelligence and statistics, San Juan, Puerto Rico

  10. Chang H, Yeung D, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, Washington, DC, USA, pp 275–282

  11. Dollar P, Rabaud V, Belongie S (2007) Learning to traverse image manifolds. In: Advances in neural information processing systems, p 19

  12. Donoho DL, Grimes C (2003) Hessian eigenmaps: locally linear embedding techniques for highdimensional data. Proc Natl Acad Arts Sci 100(10): 5591–5596

    Article  MATH  MathSciNet  Google Scholar 

  13. Duchon J (1977) Splines minimizing rotation-invariant semi-norms in sobolev spaces. In: Dold A, Eckmann B (eds) Constructive theory of functions of several variables. Springer, Heidelberg, pp 85–100

    Chapter  Google Scholar 

  14. Golub GH, van Loan CF (1996) Matrix computations. The Johns Hopkins University Press, Maryland

    MATH  Google Scholar 

  15. Gong H, Pan C, Yang Q, Lu H, Ma S (2005) A semi-supervised framework for mapping data to the intrinsic manifold. In: Proceedings of international conference on computer vision, Beijing, China, pp 98–105

  16. Gorban A, Kegl B, Wunsch D, Zinovyev A (2007) Principal manifolds for data visualization and dimension reduction. Springer, Berlin

    Book  Google Scholar 

  17. Gourier N, Hall D, Crowley J (2004) Estimating face orientation from robust detection of salient facial features. In: ICPR workshop on visual observation of deictic gestures

  18. Hadid A, Kouropteva O, Pietikainen M (2002) Unsupervised learning using locally linear embedding: experiments in face pose analysis. In: International conference of pattern recognition, vol 1, Quebec, Canada, pp 111–114

  19. Ham J, Lee DD, Saul LK (2004) Semisupervised alignment of manifolds. In: International workshop on artificial intelligence and statistics Barbados, West Indies, pp 120–127

  20. Hastie T, Stuetzle W (1989) Principal curves. J Am Stat Assoc 84(406): 502–516

    Article  MATH  MathSciNet  Google Scholar 

  21. He XF, Niyogi P (2003) Locality preserving projections. In: NIPS, Vancouver, Canada

  22. Hein M, Maier M (2007) Manifold denosing. In: Advances in neural information processing systems, vol 19. MIT Press, Cambridge, pp 1–8

  23. Jenkins OC, Mataric MJ (2004) A spatio-temporal extension to isomap nonlinear dimension reduction. In: ‘ICML’, Banff, Alberta, Canada, pp 441–448

  24. Jolliffe IT (1986) Principal component analysis. Springer, New York

    Google Scholar 

  25. Kalousis A, Prados J, Hilario M (2007) Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl Inf Syst 12(1): 95–116

    Article  Google Scholar 

  26. Kegl B, Krzyzak A, Linder T, Zeger K (2000) Learning and design of principal curves. IEEE Trans Pattern Anal Mach Intell 22(3): 281–297

    Article  Google Scholar 

  27. Kohonen T (2001) Self-organization maps, 3rd edn. Springer, Berlin

    Google Scholar 

  28. Kouropteva O, Okun O, Pietikäinen M (2005) Incremental locally linear embedding. Pattern Recogn 38(10): 1764–1767

    Article  MATH  Google Scholar 

  29. Law M, Jain AK (2006) Incremental nonlinear dimensionality reduction by manifold learning. IEEE Trans Pattern Anal Mach Intell 28(3): 377–391

    Article  Google Scholar 

  30. Lei H, Govindaraju V (2007) Generalized regression model for sequence matching and clustering. Knowl Inf Syst 12(1): 77–94

    Article  Google Scholar 

  31. Mao J, Jain AK (1995) Artificial neural networks for feature extraction and multivariate data projection. IEEE Trans Neural Netw 16(2): 296–317

    Google Scholar 

  32. Min W, Lu K, He X (2004) Locality pursuit embedding. Pattern Recogn 37(4): 781–788

    Article  MATH  Google Scholar 

  33. Mosci S, Rosasco L, Verri A (2007) Dimensionality reduction and generalization. In: Proceedings of international conference on machine learning, Corvallis, OR, USA, pp 657–664

  34. Nene S, Nayar S, Murase H (1996) Columbia object image library (coil-20), Technical report, Columbia University

  35. Niskanen M, Silven O (2003) Comparison of dimensionality reduction methods for wood surface inspection. In: Proceedings of international conference on quality control by artificial vision, Gatlinbura, Tennessee, USA, pp 178–188

  36. Qiu G, Guan J (2005) Color by linear neighborhood embedding. In: IEEE international conference on image processing, Genoa, Italy, pp 988–991

  37. Rosset S, Perlich C, Zadrozny B (2007) Ranking-based evaluation of regression models. Knowl Inf Syst 12(3): 331–353

    Article  Google Scholar 

  38. Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(1): 2323–2326

    Article  Google Scholar 

  39. Sammom JW (1969) A nonlinear mapping for data structure analysis. IEEE Trans Comp 18(5): 401–409

    Article  Google Scholar 

  40. Schoelkopf B, Smola AJ, Muller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5): 1299–1319

    Article  Google Scholar 

  41. Sha F, Saul LK (2005) Analysis and extension of spectral methods for nonlinear dimensionality reduction. In: Proceedings of international conference on machine learning, Bonn, Germany, pp 784–791

  42. Smola AJ, Mika S, Schoelkopf B, Williamson RC (2001) Regularized principal manifolds. J Mach Learn 1(3): 179–209

    Article  MATH  MathSciNet  Google Scholar 

  43. Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(1): 2319–2323

    Article  Google Scholar 

  44. Tibshirani R (1992) Principal curves revisited. Stat Comput 2: 183–190

    Article  Google Scholar 

  45. Vlachos M, Domeniconi C, Gunopulos D (2002) Non-linear dimensionality reduction techniques for classification and visualization. In: International conference on knowledge discovery and data mining, Edmonton, Canada, pp 645–651

  46. Wahba G (1990) Spline models for observational data. SIAM Press, Philadelphia

    MATH  Google Scholar 

  47. Weinberger KQ, Sha F, Saul LK (2004) Learning a kernel matrix for nonlinear dimensionality reduction, In: Proceedings of international conference on machine learning, Banff, Canada, pp 888–905

  48. Wu X, Kumar V, Quinlan R, Quinlan JR, Ghosh J, Yang Q, Motoda H, McLachlan GJ, Ng A, Liu B, Yu PS, Zhou Z-H, Steinbach M, Hand DJ, Steinberg D (2008) Top 10 algorithms in data mining. Knowl Inf Syst 14(1): 1–37

    Article  Google Scholar 

  49. Xiang SM, Nie FP, Zhang CS, Zhang CX (2006) Spline embedding for nonlinear dimensionality reduction. In: European conference on machine learning, Berlin, Germany, pp 825–832

  50. Yan S, Xu D, Zhang B, Zhang H (2005) Graph embedding: a general framework for dimensionality reduction. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, San Diego, CA, USA, pp 830–837

  51. Yang M-H (2002) Face recognition using extended isomap. In: Proceedings of IEEE international conference on image processing, Singapore, pp 117–120

  52. Yang X, Fu H, Zha H, Barlow J (2006) Semi-supervised nonlinear dimensionality reduction. In: International conference on machine learning, Pittsburgh, USA, pp 1065–1072

  53. Yoon J (2001) Spectral approximation orders of radial basis function interpolation on the sobolev space. SIAM J Math Anal 33(4): 946–958

    Article  MATH  MathSciNet  Google Scholar 

  54. Zhang Z, Zha H (2004) Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J Sci Comput 26(1): 313–338

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tsinghua National Laboratory for Information Science and Technology (TNList), Department of Automation, Tsinghua University, 3-120 Room, FIT Building, Beijing, 100084, People’s Republic of China

    Shiming Xiang, Feiping Nie, Yangqiu Song & Changshui Zhang

  2. Software School, School of Computer Science, Beijing Institute of Technology, Beijing, 100081, China

    Chunxia Zhang

Authors
  1. Shiming Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feiping Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yangqiu Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Changshui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chunxia Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiming Xiang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xiang, S., Nie, F., Song, Y. et al. Embedding new data points for manifold learning via coordinate propagation. Knowl Inf Syst 19, 159–184 (2009). https://doi.org/10.1007/s10115-008-0161-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-008-0161-3

Keywords

Search

Navigation