Skip to main content
SpringerLink
Log in

Nonlinear dimensionality reduction and data visualization: A review

  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

Abstract

Dimensionality reduction and data visualization are useful and important processes in pattern recognition. Many techniques have been developed in the recent years. The self-organizing map (SOM) can be an efficient method for this purpose. This paper reviews recent advances in this area and related approaches such as multidimensional scaling (MDS), nonlinear PCA, principal manifolds, as well as the connections of the SOM and its recent variant, the visualization induced SOM (ViSOM), with these approaches. The SOM is shown to produce a quantized, qualitative scaling and while the ViSOM a quantitative or metric scaling and approximates principal curve/surface. The SOM can also be regarded as a generalized MDS to relate two metric spaces by forming a topological mapping between them. The relationships among various recently proposed techniques such as ViSOM, Isomap, LLE, and eigenmap are discussed and compared.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. E. Condon, B. Golden, S. Lele, S. Raghavan, E. Wasil. A Visualization Model Based on Adjacency Data. Decision Support Systems, vol. 33, no. 4, pp. 349–362, 2002.

    Article  Google Scholar 

  2. H. Ni, H. Yin. Recurrent Self-organizing Maps and Local Support Vector Machine Models for Exchange Rate Prediction, In Proceedings of International Symposium on Neural Networks, Chengdu, China, vol. 3, pp. 504–511, 2006.

    Google Scholar 

  3. R. Freeman, H. Yin. Web Content Management by Self-organization. IEEE Transactions on Neural Networks, vol. 16, no. 5, pp. 1256–1268, 2005.

    Article  Google Scholar 

  4. T. Kohonen, S. Kaski, K. Lagus, J. Salojarvi, J. Honkela, V. Paetero, A. Saerelo. Self-organization of Massive Text Document Collection, in Kohonen Maps, E. Oja and S. Kaski (eds.), pp. 171–182, 1999.

  5. P. Toronen, K. Kolehmainen, G. Wong, E. Castren. Analysis of Gene Expression Data Using Self-organizing Maps. FEBS Letters, vol. 451, no. 2, pp. 142–146, 1999.

    Article  Google Scholar 

  6. E. C. Malthouse. Limitations of Nonlinear PCA as Performed with Generic Neural Networks. IEEE Transactions on Neural Networks, vol. 9, no. 1, pp. 165–173, 1998.

    Article  Google Scholar 

  7. M. A. Kramer. Nonlinear Principal Component Analysis Using Autoassociative Neural Networks. AIChE Journal, vol. 37, no. 2, pp. 233–243, 1991.

    Article  Google Scholar 

  8. J. Karhunen, J. Joutsensalo. Generalization of Principal Component Analysis, Optimization Problems, and Neural Networks. Neural Networks, vol. 8, no. 4, pp. 549–562, 1995. [51] B.

    Article  Google Scholar 

  9. B. Schőlkopf, A. Smola, K. R. Müller, Nonlinear Component Analysis as a Kernel Eigenvalue Problem, Neural Computation, vol. 10, no. 5, pp. 1299–1319, 1998.

    Article  Google Scholar 

  10. T. Hastie, W. Stuetzle. Principal Curves. Journal of the American Statistical Association, vol. 84, no. 406, pp. 502–516, 1989.

    Article  MATH  MathSciNet  Google Scholar 

  11. M. LeBlanc, R. J. Tibshirani. Adaptive Principal Surfaces. Journal of the American Statistical Association, vol. 89, no. 425, pp. 53–64, 1994.

    Article  MATH  Google Scholar 

  12. J. B. Tenenbaum, V. de Silva, J. C. Langford. A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science, vol. 290, pp. 2319–2323, 2000.

    Article  Google Scholar 

  13. S. T. Roweis, L.K. Saul. Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science, vol. 290, pp. 2323–2326, 2000.

    Article  Google Scholar 

  14. T. F. Cox, M. A. A. Cox, Multidimensional Scaling, Chapman & Hall, 1994.

  15. B. D. Ripley. Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, 1996.

    MATH  Google Scholar 

  16. J. W. Sammon. A Nonlinear Mapping for Data Structure Analysis. IEEE Transactions on Computer, vol. C-18, no. 5, pp. 401–409, 1969.

    Article  Google Scholar 

  17. J. Mao, A. K. Jain. Artificial Neural Networks for Feature Extraction and Multivariate Data Projection. IEEE Transactions on Neural Networks, vol. 6, no. 2, pp. 296–317, 1995.

    Article  Google Scholar 

  18. D. Lowe, M. E. Tipping. Feed-forward Neural Networks and Topographic Mappings for Exploratory Data Analysis. Neural Computing & Applications, vol. 4, no. 2, pp. 83–95, 1996.

    Article  Google Scholar 

  19. T. Kohonen. Self-organized Formation of Topologically Correct Feature Map. Biological Cybernetics, vol. 43, no. 1, pp. 56–69, 1982.

    Article  MathSciNet  Google Scholar 

  20. T. Kohonen. Self-organizing Maps, Springer-Verlag, Berlin, 1997.

    MATH  Google Scholar 

  21. H. Yin. ViSOM-A Novel Method for Multivariate Data Projection and Structure Visualization. IEEE Transactions on Neural Networks, vol. 13, no. 1, pp. 237–243, 2002.

    Article  Google Scholar 

  22. C. von der Malsburg, D. J. Willshaw. Self-organization of Orientation Sensitive Cells in the Striate Cortex. Biological Cybernetic, vol. 14, no. 2, pp. 85–100, 1973.

    Google Scholar 

  23. D. J. Willshaw, C. von der Malsburg. How Patterned Neural Connections Can be Set Up by Self-Organization. Proceedings of Royal Society of London, Series B, Biological Sciences, vol. 194, no. 1117, pp. 431–445, 1976.

    Article  Google Scholar 

  24. T. Kohonen. Self-organization and Associative Memory, Springer-Verlag, Berlin, 1984.

    MATH  Google Scholar 

  25. T. Kohonen. Representation of Sensory Information in Self-Organizing Feature Maps, and Relation of These Maps to Distributed Memory Networks, Proceedings of SPIE, vol. 634, pp. 248–259, 1986.

    Google Scholar 

  26. H. Yin. Self-Organizing Maps: Statistical Analysis, Treatment and Applications, Ph.D. dissertation, University of York, UK, 1996.

    Google Scholar 

  27. H. Yin, N. M. Allinson. On the Distribution and Convergence of the Feature Space in Self-organizing Maps, Neural Computation, vol. 7, pp. 1178–1187, 1995.

    Google Scholar 

  28. S. P. Luttrell. Derivation of a Class of Training Algorithms. IEEE Transactions on Neural Networks, vol. 1, no. 2, pp. 229–232, 1990.

    Article  Google Scholar 

  29. S. P. Luttrell. A Bayesian Analysis of Self-organizing Maps. Neural Computation, vol. 6, no. 5, pp. 767–794, 1994.

    MATH  Google Scholar 

  30. H. Ritter, K. Schulten. Convergence Properties of Kohonen’s Topology Conserving Maps: Fluctuations, Stability, and Dimension Selection. Biological Cybernetics, vol. 60, no. 1, pp. 59–71, 1988.

    Article  MATH  MathSciNet  Google Scholar 

  31. E. Erwin, K. Obermayer, K. Schulten. Self-organizing Maps: Ordering, Convergence Properties and Energy Functions. Biological Cybernetics, vol. 67, no. 1, pp. 47–55, 1992.

    Article  MATH  Google Scholar 

  32. E. Erwin, K. Obermayer, K. Schulten. Self-organizing Maps: Stationary States, Metastability and Convergence Rate. Biological Cybernetics, vol. 67, no. 1, pp. 35–45, 1992.

    Article  MATH  Google Scholar 

  33. Z.P. Lo, B. Bavarian. On the Rate of Convergence in Topology Preserving Neural Networks. Biological Cybernetics, vol. 65, no. 1, pp. 55–63, 1991.

    Article  MATH  MathSciNet  Google Scholar 

  34. S. Lin, J. Si. Weight-value Convergence of the SOM Algorithm for Discrete Input. Neural Computation, vol. 10, no. 4, pp. 807–814, 1998.

    Article  Google Scholar 

  35. E. de Bolt, M. Cottrell, M. Verleysen. Statistical Tools to Assess the Reliability of Self-organising Maps. Neural Networks, vol. 15, no. 8–9, pp. 967–978, 2002.

    Google Scholar 

  36. R. Durbin, G. Mitchison. A Dimension Reduction Framework for Understanding Cortical Maps. Nature, vol. 343, no. 6259, pp. 644–647, 1990.

    Article  Google Scholar 

  37. G. Mitchison. A Type of Duality Between Self-organizing Maps and Minimal Wiring. Neural Computation, vol. 7, no. 1, pp. 25–35, 1995.

    Article  Google Scholar 

  38. T. Kohonen. Self-organizing Maps: Optimization Approaches. Artificial Neural Networks, vol. 2, no. 5, pp. 981–990, 1991.

    Google Scholar 

  39. J. Lampinen, E. Oja. Clustering Properties of Hierarchical Selforganizing Maps. Journal of Mathematical Imaging and Vision, vol. 2, no. 2–3, pp. 261–272, 1992.

    Article  MATH  Google Scholar 

  40. T. Heskes. Energy Functions for Self-organizing Maps. Kohonen Maps, E. Oja and S. Kaski (eds.), pp. 303–315, 1999.

  41. G. J. Goodhill, T. Sejnowski. A Unifying Objective Function for Topographic Mappings. Neural Computation, vol. 9, no. 6, pp. 1291–1303, 1997.

    Article  Google Scholar 

  42. H.-U. Bauer, K. R. Pawelzik. Quantifying the Neighborhood Preservation of Self-organizing Feature Maps. IEEE Transactions on Neural Networks, vol. 3, no. 4, pp. 570–579, 1992.

    Article  Google Scholar 

  43. T. Villmann, R. Der, M. Herrmann, T. M. Martinetz. Topology Preservation in Self-organizing Feature Maps: Exact Definition and Measurement. IEEE Transactions on Neural Networks, vol. 8, no. 2, pp. 256–266, 1997.

    Article  Google Scholar 

  44. S. Lawrence, C. L. Giles, A. C. Tsoi, A. D. Back. Face Recognition: A Convolutional Neural-network Approach. IEEE Transactions on Neural Networks, vol. 8, no. 1, pp. 98–113, 1997.

    Article  Google Scholar 

  45. H. Ritter, T. Martinetz, K. Schulten. Neural Computation and Self-organizing Maps: An Introduction, Addison Wesley Publishing Company, Danvers, 1992.

    MATH  Google Scholar 

  46. H. Yin, N. M. Allinson. Interpolating Self-organizing Map (iSOM). Electronics Letters, vol. 35, no. 19, pp. 1649–1650, 1999.

    Article  Google Scholar 

  47. H. Yin. Resolution Enhancement for the ViSOM, In Proceedings of the Workshop on Self-organizing Maps, Kitakyushu, Japan, pp. 208–212. 2003.

  48. H. Yin. Data Visualization and Manifold Mapping Using the ViSOM. Neural Networks, vol. 15, no. 8–9, pp. 1005–1016, 2002.

    Article  Google Scholar 

  49. S. Wu, T. W. S. Chow. PRSOM: A New Visualization Method by Hybridizing Multidimensional Scaling and Self-organizing Map. IEEE Transactions on Neural Networks, vol. 16, no. 6, pp. 1362–1380, 2005.

    Article  Google Scholar 

  50. P. A. Estévez, C. J. Figueroa. Online Data Visualization Using the Neural Gas Network. Neural Networks, vol. 19, no. 6–7, pp. 923–934, 2006.

    Article  MATH  Google Scholar 

  51. R. C. T. Lee, J. R. Slagle, H. Blum. A Triangulation Method for the Sequential Mapping of Points from N-Space to Two-Space. IEEE Transactions on Computers, vol. C-26, no. 3, pp. 288–292, 1977.

    Google Scholar 

  52. D. de Ridder, R. P. W. Duin. Sammon’s Mapping Using Neural Networks: A Comparison. Pattern Recognition Letters, vol. 18, no. 11–13, pp. 1307–1316, 1997.

    Article  Google Scholar 

  53. M. Balasubramanian, E. L. Schwartz. The Isomap Algorithm and Topological Stability. Science, vol. 295, no. 5552, pp. 7, 2002.

    Article  Google Scholar 

  54. E. Oja. Neural Networks, Principal Components, and Subspaces. International Journal of Neural Systems, vol. 1, no. 1, pp. 61–68, 1989.

    Article  MathSciNet  Google Scholar 

  55. T. D. Sanger. Optimal Unsupervised Learning in a Single-layer Linear Feedforward Network. Neural Networks, vol. 2, no. 6, pp. 459–473, 1989.

    Article  Google Scholar 

  56. M. Belkin, P. Niyogi. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation, vol. 15, no. 6, pp. 1373–1396, 2003.

    Article  MATH  Google Scholar 

  57. Y. Weiss. Segmentation Using Eigenvectors: A Unifying View. In Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece, pp. 975–982, 1999.

  58. R. Tibshirani. Principal Curves Revisited. Statistics and Computation, vol. 2, no. 4, pp. 183–190, 1992.

    Article  Google Scholar 

  59. Y. Linde, A. Buzo, R. M. Gray. An Agorithm for Vctor Qantizer Dsign. IEEE Transactions on Communications, vol. 28, no. 1, pp. 84–95, 1980.

    Article  Google Scholar 

  60. J. D. Banfield, A. E. Raftery. Ice Floe Identification in Satellite Images Using Mathematical Morphology and Clustering about Principal Curves. Journal of the American Statistical Association, vol. 87, no. 417, pp. 7–16, 1992.

    Article  Google Scholar 

  61. B. Kegl, A. Krzyzak, T. Linder, K. Zeger. A Polygonal Line Algorithm for Constructing Principal Curves, In Proceeding of Neural Information Processing Systems, Computer Press, Denver Colorado, USA, pp. 501–507, 1998.

    Google Scholar 

  62. C. M. Bishop, M. Svensn, C. K. I. Williams. GTM: The Generative Topographic Mapping. Neural Computation, vol. 10, no. 1, pp. 215–235, 1998.

    Article  Google Scholar 

  63. K. Y. Chang, J. Ghosh. A Unified Model for Probabilistic Principal Surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 1, pp. 22–41, 2001.

    Article  Google Scholar 

  64. A. Flexer. Limitations of Self-organizing Maps for Vector Quantization and Multidimensional Scaling, In Proceedings of Neural Information Processing Systems, Computer Press, Denver Colorado, USA, pp. 445–451, 1997.

    Google Scholar 

  65. H. Yin. Connection Between Self-organizing Maps and Metric Multidimensional Scaling, In Proceedings of International Joint Conference on Neural Networks, Orlando, Florida, USA, pp. 12–17, 2007.

  66. P. Delicado. Another Look at Principal Curves and Surfaces. Journal of Multivariate Analysis. vol. 77, no. 1, pp. 84–116, 2001.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, The University of Manchester, Manchester, M60 1QD, UK

    Hujun Yin

Authors
  1. Hujun Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Hujun Yin Senior lecturer at the University of Manchester, School of Electrical and Electronic Engineering. He received his B.Eng., M.Sc. from Southeast University and Ph.D. from University of York in 1983, 1986 and 1996 respectively. His research interests include neural networks, self-organizing systems in particular, pattern recognition, image processing and bioinformatics.

Dr. Yin has studied, extended and applied the self-organizing map (SOM) and related topics (principal manifolds and data visualization) extensively in the last ten years and proposed a number of extensions including the Bayesian SOM and ViSOM. He has served on the Programme Committee for more than twenty international conferences. He was the Organizing and Programme Committee Chair and General Chair for a number of conferences, including, International Workshop on Self-Organizing Maps (WSOM’01), International Conference on Intelligent Data Engineering and Automated Learning (IDEAL’03-IDEAL’07), and International Symposium on Neural Networks (ISNN’04-ISNN’06). He sits on the Steering Committee of the WSOM series. He was a guest editor of Neural Networks: 2002 Special Issue on New Developments in Self-Organizing Maps, among two other special issues on two other international journals. He has received research funding from the EPSRC, BBSRC and DTI.

Dr. Yin has published more than 90 peer-reviewed articles. He is a senior member of the IEEE and a member of the EPSRC Peer Review College. He has also been a regular referee for the EPSRC, BBSRC and Royal Society grant proposals, Hong Kong Research Grant Councils, etherlands Organization for Scientific Research, and Slovakia Research and Development Council. He is an associate editor of the IEEE Transactions on Neural Networks and a member of the Editorial Board of the International Journal of Neural Systems.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yin, H. Nonlinear dimensionality reduction and data visualization: A review. Int J Automat Comput 4, 294–303 (2007). https://doi.org/10.1007/s11633-007-0294-y

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-007-0294-y

Keywords

Search

Navigation