Skip to main content
SpringerLink
Log in

Nonlinear Principal Manifolds – Adaptive Hybrid Learning Approaches

  • Conference paper
Hybrid Artificial Intelligence Systems (HAIS 2008)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5271))

Included in the following conference series:

Abstract

Dimension reduction has long been associated with retinotopic mapping for understanding cortical maps. Multisensory information is processed, fused, fed and mapped to a 2-D cortex in a near-optimal information preserving manner. Data projection and visualization, inspired by this mechanism, are playing an increasingly important role in many computational applications such as cluster analysis, classification, data mining, knowledge management and retrieval, decision support, marketing, image processing and analysis. Such tasks involving either visual and spatial analysis or reduction of features or volume of the data are essential in many fields from biology, neuroscience, decision support, to management science. The topic has recently attracted a great deal of attention. There have been considerable advances in methodology and techniques for extracting nonlinear manifold as to reduce data dimensionality and a number of novel methods have been proposed from statistics, geometry theory and adaptive neural networks. Typical approaches include multidimensional scaling, nonlinear PCA and principal curve/surface. This paper provides an overview on this challenging and emerging topic. It discusses various recent methods such as self-organizing maps, kernel PCA, principal manifold, isomap, local linear embedding, Laplacian eigenmap and spectral clustering, and many of them can be seen as a combined, adaptive learning framework. Their usefulness and potentials will be presented and illustrated in various applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Balasubramanian, M., Schwartz, E.L.: The Isomap algorithm and topological stability. Science 295, 7a (2002)

    Article  Google Scholar 

  2. Banfield, J.D., Raftery, A.E.: Ice floe identification in satellite images using mathe-matical morphology and clustering about principal curves. J. Amer. Statist. Assoc. 87, 7–16 (1992)

    Article  Google Scholar 

  3. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15, 1373–1396 (2003)

    Article  MATH  Google Scholar 

  4. Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: The generative topographic mapping. Neural Computation 10, 215–235 (1998)

    Article  Google Scholar 

  5. Borg, I., Groenen, P.J.F.: Modern Multidimensional Scaling: Theory and Applications, 2nd edn. Springer, Heidelberg (2005)

    MATH  Google Scholar 

  6. Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching. PNAS 103, 1168–1172 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  7. Chang, K.-Y., Ghosh, J.: A unified model for probabilistic principal surfaces. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 22–41 (2001)

    Article  Google Scholar 

  8. Cox, T.F., Cox, M.A.A.: Multidimensional Scaling. Chapman & Hall, London (1994)

    MATH  Google Scholar 

  9. De Ridder, D., Duin, R.P.W.: Sammon mapping using neural networks: A comparison. Pattern Recognition Letters 18, 1307–1316 (1997)

    Article  Google Scholar 

  10. Durbin, R., Mitchison, G.: A dimension reduction framework for understanding cortical maps. Nature 343, 644–647 (1990)

    Article  Google Scholar 

  11. Estévez, P.A., Figueroa, C.J.: Online data visualization using the neural gas network. Neural Networks 19, 923–934 (2006)

    Article  MATH  Google Scholar 

  12. Flexer, A.: Limitations of self-organizing maps for vector quantization and multidimensional scaling. Advances in Neural Information Processing Systems 10, 445–451 (1997)

    Google Scholar 

  13. Freeman, R.T., Yin, H.: Adaptive topological tree structure for document organisation and visualisation. Neural Networks 17, 1255–1271 (2004)

    Article  MATH  Google Scholar 

  14. Fyfe, C.: Two topographic maps for data visualisation. Data Mining and Knowledge Discovery 14, 207–224 (2007)

    Article  MathSciNet  Google Scholar 

  15. Goodhill, G.J., Sejnowski, T.: A unifying objective function for topographic mappings. Neural Computation 9, 1291–1303 (1997)

    Article  Google Scholar 

  16. Gorban, A.N., Kégl, B., Wunsch, D.C., Zinovyev, A.: Principal Manifolds for Data Visualization and Dimension Reduction. Springer, Heidelberg (2008)

    MATH  Google Scholar 

  17. Gorban, A., Zinovyev, A.: Method of elastic maps and its applications in data visualization and data modeling. Int. J. Computing Anticipatory Systems 12, 353–369 (2001)

    Google Scholar 

  18. Ham, J., Lee, D.D., Mika, S., Schölkopf, B.: A kernel view of the dimensionality reduction of manifolds. In: Proc. 21st Int. Conf. on Machine Learning, p. 47 (2004)

    Google Scholar 

  19. Hastie, T., Stuetzle, W.: Principal curves. J. Amer. Statist. Assoc. 84, 502–516 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  20. He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. Advances in Neural Information Processing Systems 18 (2005)

    Google Scholar 

  21. Honkela, T., Kaski, S., Lagus, K., Kohonen, T.: WEBSOM – self-organizing maps of document collections. In: Proc. Workshop on Self-Organizing Maps, pp. 310–315 (1997)

    Google Scholar 

  22. Karhunen, J., Joutsensalo, J.: Generalisation of principal component analysis, optimisation problems, and neural networks. Neural Networks 8, 549–562 (1995)

    Article  Google Scholar 

  23. Kégl, B., Krzyzak, A., Linder, T., Zeger, K.: A polygonal line algorithm for constructing principal curves. Advances in Neural Information Processing Systems 11, 501–507 (1998)

    Google Scholar 

  24. Kohonen, T.: Self-organised formation of topologically correct feature map. Biological Cybernetics 43, 56–69 (1982)

    Article  MathSciNet  Google Scholar 

  25. Kohonen, T.: Self-organization and associative memory. Springer, Heidelberg (1984)

    MATH  Google Scholar 

  26. Kohonen, T.: The adaptive-subspace SOM (ASSOM) and its use for the implementation of invariant feature detection. In: Proc. Int. Conf. on Artificial Neural Networks, pp. 3–10 (1995)

    Google Scholar 

  27. Kohonen, T.: Self-Organising Maps, 2nd edn. Springer, Heidelberg (1997)

    Google Scholar 

  28. Kramer, M.A.: Nonlinear principal component analysis using autoassociative neural networks. AICHE (American Institute of Chemical Engineers) Journal 37, 233–243 (1991)

    Google Scholar 

  29. Lawrence, S., Giles, C.L., Tsoi, A.C., Back, A.D.: Face recognition: A convolutional neuralnetwork approach. IEEE Trans. Neural Networks 8, 98–113 (1997)

    Article  Google Scholar 

  30. LeBlanc, M., Tibshirani, R.J.: Adaptive principal surfaces. J. Amer. Statist. Assoc. 89, 53–64 (1994)

    Article  MATH  Google Scholar 

  31. Lee, R.C.T., Slagle, J.R., Blum, H.: A triangulation method for the sequential mapping of points from n-space to two-space. IEEE Trans. Computers 27, 288–292 (1977)

    Article  Google Scholar 

  32. Lowe, D., Tipping, M.E.: Feed-forward neural networks and topographic mappings for exploratory data analysis. Neural Computing and Applications 4, 83–95 (1996)

    Article  Google Scholar 

  33. Luttrell, S.P.: Derivation of a class of training algorithms. IEEE Trans. Neural Networks 1, 229–232 (1990)

    Article  Google Scholar 

  34. Luttrell, S.P.: A Bayesian analysis of self-organising maps. Neural Computation 6, 767–794 (1994)

    Article  MATH  Google Scholar 

  35. Malthouse, E.C.: Limitations of nonlinear PCA as performed with generic neural networks. IEEE Trans. Neural Networks 9, 165–173 (1998)

    Article  Google Scholar 

  36. Mao, J., Jain, A.K.: Artificial neural networks for feature extraction and multivariate data projection. IEEE Trans. Neural Networks 6, 296–317 (1995)

    Article  Google Scholar 

  37. Mitchison, G.: A type of duality between self-organizing maps and minimal wiring. Neural Computation 7, 25–35 (1995)

    Article  Google Scholar 

  38. Möller-Levet, C.S., Yin, H.: Modeling and analysis of gene expression time-series based on co-expression. Int. J. Neural Systems 15, 311–322 (2005)

    Article  Google Scholar 

  39. Nguyen, D.V., Rockeb, D.M.: On partial least squares dimension reduction for microarray-based classification: a simulation study. Computational Statistics & Data Analysis 46, 407–425 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  40. Ni, H., Yin, H.: Time-series prediction using self-organising mixture autoregressive network. In: Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds.) IDEAL 2007. LNCS, vol. 4881, pp. 1000–1009. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  41. Oja, E.: Neural networks, principal components, and subspaces. Int. J. Neural Systems 1, 61–68 (1989)

    Article  MathSciNet  Google Scholar 

  42. Ripley, B.D.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (1996)

    MATH  Google Scholar 

  43. Ritter, H., Martinetz, T., Schulten, K.: Neural Computation and Self-organising Maps: An Introduction. Addison-Wesley Publishing Company, Reading (1992)

    Google Scholar 

  44. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)

    Article  Google Scholar 

  45. Sammon, J.W.: A nonlinear mapping for data structure analysis. IEEE Trans. Computer 18, 401–409 (1969)

    Article  Google Scholar 

  46. Sanger, T.D.: Optimal unsupervised learning in a single-layer linear feedforward network. Neural Networks 2, 459–473 (1991)

    Article  Google Scholar 

  47. Schölkopf, B., Smola, A., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)

    Article  Google Scholar 

  48. Smola, A.J., Williamson, R.C., Mika1, S., Schölkopf, B.: Regularized principal manifolds. In: Fischer, P., Simon, H.U. (eds.) EuroCOLT 1999. LNCS (LNAI), vol. 1572, pp. 214–229. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  49. Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)

    Article  Google Scholar 

  50. Tibshirani, R.: Principal curves revisited. Statistics and Computation 2, 183–190 (1992)

    Article  Google Scholar 

  51. Weiss, Y.: Segmentation using eigenvectors: a unified view. In: Proc. IEEE Int. Conf. on Computer Vision, pp. 975–982 (1999)

    Google Scholar 

  52. Willshaw, D.J., von der Malsburg, C.: How patterned neural connections can be set up by self-organization. Proc. Royal Society of London Series B 194, 431–445 (1976)

    Article  Google Scholar 

  53. Wu, S., Chow, T.W.S.: PRSOM: A new visualization method by hybridizing multidimensional scaling and self-organizing map. IEEE Trans. Neural Networks 16, 1362–1380 (2005)

    Article  Google Scholar 

  54. Yin, H.: ViSOM-A novel method for multivariate data projection and structure visualisation. IEEE Trans. Neural Networks 13, 237–243 (2002)

    Article  Google Scholar 

  55. Yin, H.: Data visualisation and manifold mapping using the ViSOM. Neural Networks 15, 1005–1016 (2002)

    Article  Google Scholar 

  56. Yin, H.: Nonlinear multidimensional data projection and visualisation. In: Liu, J., Cheung, Y.-m., Yin, H. (eds.) IDEAL 2003. LNCS, vol. 2690, pp. 377–388. Springer, Heidelberg (2003)

    Google Scholar 

  57. Yin, H.: Resolution enhancement for the ViSOM. In: Proc. Workshop on Self-Organizing Maps, Kitakyushu, Japan, Kyushu Institute of Technology, pp. 208–212 (2003)

    Google Scholar 

  58. Yin, H.: Connection between self-organising maps and metric multidimensional scaling. In: Proc. Int. Joint Conf. on Neural Networks, Orlando, pp. 1025–1030. IEEE Press, Los Alamitos (2007)

    Chapter  Google Scholar 

  59. Yin, H.: On multidimensional scaling and the embedding of self-organizing maps. Neural Networks 21, 160–169 (2008)

    Article  Google Scholar 

  60. Yin, H., Panzeri, P., Mehboob, Z., Diamond, M.: Decoding population neuronal responses by topological clustering. In: Proc. Int. Conf. on Artificial Neural Networks. LNCS, vol. 5164, pp. 547–557. Springer, Heidelberg (2008)

    Google Scholar 

  61. Yin, H., Allinson, N.M.: On the distribution and convergence of the feature space in self-organising maps. Neural Computation 7, 1178–1187 (1995)

    Article  Google Scholar 

  62. Yin, H., Allinson, N.M.: Self-organising mixture networks for probability density estimation. IEEE Trans. Neural Networks 12, 405–411 (2001)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The University of Manchester, Manchester, M60 1QD, UK

    Hujun Yin

Authors
  1. Hujun Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Escuela Politécnica Superior, GICAP Research Group, Universidad de Burgo, Calle Francisco de Vitoria S/N, Edifico C, Campus Vena, 09006, Burgos, Spain

    Emilio Corchado

  2. Center of Excellence for Quantifiable Quality of Service, Norwegian University of Science and Technology, 7491, Trondheim, Norway

    Ajith Abraham

  3. Department of Electrical and Computer Engineering, University of Alberta, Edmonton,, T6G 2V4, Alberta, Canada

    Witold Pedrycz

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Yin, H. (2008). Nonlinear Principal Manifolds – Adaptive Hybrid Learning Approaches. In: Corchado, E., Abraham, A., Pedrycz, W. (eds) Hybrid Artificial Intelligence Systems. HAIS 2008. Lecture Notes in Computer Science(), vol 5271. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87656-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-87656-4_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-87655-7

  • Online ISBN: 978-3-540-87656-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation