Skip to main content
SpringerLink
Log in

Nonlinear Dimensionality Reduction by Topologically Constrained Isometric Embedding

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Many manifold learning procedures try to embed a given feature data into a flat space of low dimensionality while preserving as much as possible the metric in the natural feature space. The embedding process usually relies on distances between neighboring features, mainly since distances between features that are far apart from each other often provide an unreliable estimation of the true distance on the feature manifold due to its non-convexity. Distortions resulting from using long geodesics indiscriminately lead to a known limitation of the Isomap algorithm when used to map non-convex manifolds. Presented is a framework for nonlinear dimensionality reduction that uses both local and global distances in order to learn the intrinsic geometry of flat manifolds with boundaries. The resulting algorithm filters out potentially problematic distances between distant feature points based on the properties of the geodesics connecting those points and their relative distance to the boundary of the feature manifold, thus avoiding an inherent limitation of the Isomap algorithm. Since the proposed algorithm matches non-local structures, it is robust to strong noise. We show experimental results demonstrating the advantages of the proposed approach over conventional dimensionality reduction techniques, both global and local in nature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aharon, M., & Kimmel, R. (2006). Representation analysis and synthesis of lip images using dimensionality reduction. International Journal of Computer Vision, 67(3), 297–312.

    Article  Google Scholar 

  • Belkin, M., & Niyogi, P. (2002). Laplacian eigenmaps and spectral techniques for embedding and clustering. In Dietterich, T. G., Becker, S., & Ghahramani, Z. (Eds.), Advances in neural inf. proc. sys. (Vol. 14, pp. 585–591). Cambridge: MIT Press.

    Google Scholar 

  • Belton, D., & Lichti, D. D. (2007). Classification and segmentation of terrestrial laserscanner point clouds using local variance information. ISPRS Image Engineering and Vision Metrology, 61(5), 307–324.

    Google Scholar 

  • Ben-Tal, A., & Nemirovski, A. S. (2001). Lectures on modern convex optimization: analysis, algorithms, and engineering applications. Philadelphia: Society for Industrial and Applied Mathematics.

    MATH  Google Scholar 

  • Bernstein, M., de Silva, V., Langford, J. C., & Tenenbaum, J. B. (2001). Graph approximations to geodesics on embedded manifolds (Technical Report). Stanford University.

  • Blake, A., & Zisserman, A. (1987). Visual reconstruction. Cambridge: MIT Press.

    Google Scholar 

  • Borg, I., & Groenen, P. (1997). Modern multidimensional scaling: theory and applications. New York: Springer.

    MATH  Google Scholar 

  • Boult, T. E., & Kender, J. R. (1986). Visual surface reconstruction using sparse depth data. In Computer vision and pattern recognition (Vol. 86, pp. 68–76).

  • Brand, M. (2005). Nonrigid embeddings for dimensionality reduction. In Gama, J., Camacho, R., Brazdil, P., Jorge, A., & Torgo, L. (Eds.), Lecture notes in computer science : Vol. 3720. ECML (pp. 47–59). Berlin: Springer.

    Google Scholar 

  • Brandes, U., & Pich, C. (2007). Eigensolver methods for progressive multidimensional scaling of large data. In Kaufmann, M., & Wagner, D. (Eds.), Graph drawing, Karlsruhe, Germany, September 18–20, 2006 (pp. 42–53). Berlin: Springer.

    Chapter  Google Scholar 

  • Bronstein, A. M., Bronstein, M. M., & Kimmel, R. (2005). Three-dimensional face recognition. International Journal of Computer Vision, 64(1), 5–30.

    Article  Google Scholar 

  • Bronstein, A. M., Bronstein, M. M., & Kimmel, R. (2006a). Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching. Proceedings of the National Academy of Science of the USA, 103(5), 1168–1172.

    Article  MATH  MathSciNet  Google Scholar 

  • Bronstein, M. M., Bronstein, A. M., Kimmel, R., & Yavneh, I. (2006b). Multigrid multidimensional scaling. Numerical Linear Algebra with Applications, 13(2–3), 149–171. Special issue on multigrid methods.

    Article  MATH  MathSciNet  Google Scholar 

  • Cabay, S., & Jackson, L. (1976). A polynomial extrapolation method for finding limits and antilimits of vector sequences. SIAM Journal on Numerical Analysis, 13, 734–752.

    Article  MATH  MathSciNet  Google Scholar 

  • Chalmers, M. (1996). A linear iteration time layout algorithm for visualising high-dimensional data. In IEEE visualization (pp. 127–132).

  • Chazal, F., Cohen-Steiner, D., & Mérigot, Q. (2007). Stability of boundary measures (Research Report 6219). INRIA Sophia Antipolis.

  • Choi, H., & Choi, S. (2007). Robust kernel isomap. Pattern Recognition, 40(3), 853–862.

    Article  MATH  Google Scholar 

  • Coifman, R. R., Lafon, S., Lee, A. B., Maggioni, M., Nadler, B., Warner, F., & Zucker, S. W. (2005). Geometric diffusions as a tool for harmonic analysis and structure definition of data. Proceedings of the National Academy of Science of the USA, 102(21), 7426–7431.

    Article  Google Scholar 

  • Cormen, T. H., Leiserson, C. E., & Rivest, R. L. (1990). Introduction to algorithms. Cambridge/New York: MIT Press/McGraw-Hill.

    MATH  Google Scholar 

  • de Leeuw, J. (1977). Applications of convex analysis to multidimensional scaling. In Barra, J., Brodeau, F., Romier, G., & Custem, B. V. (Eds.), Recent developments in statistics (pp. 133–146). Amsterdam: North Holland.

    Google Scholar 

  • de Leeuw, J. (1984). Differentiability of Kruskal’s stress at a local minimum. Psychometrika, 49(1), 111–113.

    Article  MathSciNet  Google Scholar 

  • de Leeuw, J. (1988). Convergence of the majorization method for multidimensional scaling. Journal of Classification, 5(2), 163–180.

    Article  MATH  MathSciNet  Google Scholar 

  • de Silva, V., & Tenenbaum, J. B. (2002). Global versus local methods in nonlinear dimensionality reduction. In Becker, S., Thrun, S., & Obermayer, K. (Eds.), Advances in neural inf. proc. sys (pp. 705–712). Cambridge: MIT Press.

    Google Scholar 

  • de Silva, V., & Tenenbaum, J. B. (2004). Sparse multidimensional scaling using landmark points (Tech. rep.). Stanford University.

  • Diaz, R. M., & Arencibia, A. Q. (Eds.) (2003). Coloring of DT-MRI fiber traces using Laplacian eigenmaps. Berlin: Springer.

    Google Scholar 

  • Dijkstra, E. W. (1959). A note on two problems in connection with graphs. Numerische Mathematik, 1, 269–271.

    Article  MATH  MathSciNet  Google Scholar 

  • Duda, RO, Hart, PE, & Stork, D. G. (2000). Pattern classification and scene analysis (2nd ed.). New York: Wiley.

    Google Scholar 

  • Eddy, R. P. (1979). Extrapolationg to the limit of a vector sequence. In Wang, P. C. (Ed.), Information linkage between applied mathematics and industry (pp. 387–396). San Diego: Academic Press.

    Google Scholar 

  • Elad, A., & Kimmel, R. (2003). On bending invariant signatures for surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 1285–1295.

    Article  Google Scholar 

  • Eldar, Y., Lindenbaum, M., Porat, M., & Zeevi, Y. (1997). The farthest point strategy for progressive image sampling. IEEE Transactions on Image Processing, 6(9), 1305–1315.

    Article  Google Scholar 

  • Freedman, D. (2002). Efficient simplicial reconstructions of manifolds from their samples. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(10), 1349–1357.

    Article  Google Scholar 

  • Gonzalez, T. F. (1985). Clustering to minimize the maximum intercluster distance. Theoretical Computer Science, 38, 293–306.

    Article  MATH  MathSciNet  Google Scholar 

  • Gopi, M. (2002). On sampling and reconstructing surfaces with boundaries. In Canadian conference on computational geometry (pp. 49–53).

  • Grimes, C., & Donoho, D. L. (2002). When does Isomap recover the natural parameterization of families of articulated images? (Tech. Rep. 2002-27). Department of Statistics, Stanford University, Stanford, CA 94305-4065.

  • Grimes, C., & Donoho, D. L. (2003). Hessian eigenmaps: locally linear embedding techniques for high-dimensional data. Proceedings of the National Academy of Sciences of the USA, 100(10), 5591–5596.

    Article  MATH  MathSciNet  Google Scholar 

  • Guttman, L. (1968). A general nonmetric technique for finding the smallest coordinate space for a configuration of points. Psychometrika, 33, 469–506.

    Article  MATH  Google Scholar 

  • Guy, G., & Medioni, G. (1997). Inference of surfaces, 3D curves, and junctions from sparse, noisy, 3D data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(11), 1265–1277.

    Article  Google Scholar 

  • Haque, A., & Dilts, G. A. (2007). Three-dimensional boundary detection for particle methods. Journal of Computational Physics, 226(2), 1710–1730.

    Article  MATH  MathSciNet  Google Scholar 

  • Hochbaum, D., & Shmoys, D. (1985). A best possible approximation for the k-center problem. Mathematics of Operations Research, 10(2), 180–184.

    Article  MATH  MathSciNet  Google Scholar 

  • Kearsley, A., Tapia, R., & Trosset, M. W. (1998). The solution of the metric stress and sstress problems in multidimensional scaling using Newton’s method. Computational Statistics, 13(3), 369–396.

    MATH  Google Scholar 

  • Keller, Y., Lafon, S., & Krauthammer, M. (2005). Protein cluster analysis via directed diffusion. In The fifth Georgia tech international conference on bioinformatics.

  • Kimmel, R., & Sethian, JA (1998). Computing geodesic paths on manifolds. Proceedings of the National Academy of Sciences of the USA, 95(15), 8431–8435.

    Article  MATH  MathSciNet  Google Scholar 

  • Mes̆ina, M. (1977). Convergence acceleration for the iterative solution of the equations X=AX+f. Computer Methods in Applied Mechanics and Engineering, 10, 165–173.

    Article  MathSciNet  Google Scholar 

  • Mordohai, P., & Medioni, G. (2005). Unsupervised dimensionality estimation and manifold learning in high-dimensional spaces by tensor voting. In Proceedings of international joint conference on artificial intelligence (pp. 798–803).

  • Platt, J. C. (2004). Fastmap, metricmap, and landmark MDS are all nystrom algorithms (Tech. Rep. MSR-TR-2004-26). Microsoft Research (MSR).

  • Pless, R. (2003). Using Isomap to explore video sequences. In International conference on computer vision, Nice, France (pp. 1433–1440).

  • Rosman, G., Bronstein, A. M., Bronstein, M. M., & Kimmel, R. (2006). Topologically constrained isometric embedding. In Proc. conf. on machine learning and pattern recognition (MLPR).

  • Rosman, G., Bronstein, A. M., Bronstein, M. M., Sidi, A., & Kimmel, R. (2008). Fast multidimensional scaling using vector extrapolation (Technical Report CIS-2008-01). Technnion, Israel Institute of Technology.

  • Rosman, G., Bronstein, A. M., Bronstein, M. M., & Kimmel, R. (2009). Nonlinear dimensionality reduction by topologically constrained isometric embedding (Technical Report CIS-2009-05). Technion.

  • Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323–2326.

    Article  Google Scholar 

  • Schwartz, EL, Shaw, A., & Wolfson, E. (1989). A numerical solution to the generalized mapmaker’s problem: flattening nonconvex polyhedral surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 1005–1008.

    Article  Google Scholar 

  • Sidi, A. (1991). Efficient implementation of minimal polynomial and reduced rank extrapolation methods. Journal of Computational and Applied Mathematics, 36(3), 305–337.

    Article  MATH  MathSciNet  Google Scholar 

  • Smith, D. A., Ford, W. F., & Sidi, A. (1987). Extrapolation methods for vector sequences. SIAM Review, 29(2), 199–233.

    Article  MATH  MathSciNet  Google Scholar 

  • Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500), 2319–2323.

    Article  Google Scholar 

  • Tong, W. S., Tang, C. K., Mordohai, P., & Medioni, G. (2004). First order augmentation to tensor voting for boundary inference and multiscale analysis in 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(5), 594–611.

    Article  Google Scholar 

  • Trosset, M., & Mathar, R. (1997). On the existence of nonglobal minimizers of the stress criterion for metric multidimensional scaling. In American statistical association: proceedings statistical computing section (pp. 158–162).

  • Weinberger, K. Q., & Saul, L. K. (2004). Unsupervised learning of image manifolds by semidefinite programming. In Computer vision and pattern recognition, Washington, DC (Vol. 2, pp. 988–995).

  • Weinberger, K. Q., Packer, B. D., & Saul, L. K. (2005). Nonlinear dimensionality reduction by semidefinite programming and kernel matrix factorization. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, Barbados.

  • Williams, C. K. I. (2002). On a connection between kernel PCA and metric multidimensional scaling. Machine Learning, 46(1–3), 11–19.

    Article  MATH  Google Scholar 

  • Zigelman, G., Kimmel, R., & Kiryati, N. (2002). Texture mapping using surface flattening via multidimensional scaling. IEEE Transactions on Visualization and Computer Graphics, 8(2), 198–207.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Department of Computer Science, Technion–Israel Institute of Technology, Haifa, 32000, Israel

    Guy Rosman, Michael M. Bronstein, Alexander M. Bronstein & Ron Kimmel

Authors
  1. Guy Rosman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael M. Bronstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander M. Bronstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ron Kimmel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guy Rosman.

Additional information

This research was partly supported by United States–Israel Binational Science Foundation grant No. 2004274, by the Israel Science Foundation (grant no. 623/08) by the Ministry of Science grant No. 3-3414, by the Office of Naval Research (ONR) grant, and by the Elias Fund for Medical Research.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rosman, G., Bronstein, M.M., Bronstein, A.M. et al. Nonlinear Dimensionality Reduction by Topologically Constrained Isometric Embedding. Int J Comput Vis 89, 56–68 (2010). https://doi.org/10.1007/s11263-010-0322-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-010-0322-1

Keywords

Search

Navigation