Article

Optimal dimensionality of metric space for classification

Authors:

Hong LuAuthors Info & Claims

ICML '07: Proceedings of the 24th international conference on Machine learning

Pages 1135 - 1142

https://doi.org/10.1145/1273496.1273639

Published: 20 June 2007 Publication History

Abstract

In many real-world applications, Euclidean distance in the original space is not good due to the curse of dimensionality. In this paper, we propose a new method, called Discriminant Neighborhood Embedding (DNE), to learn an appropriate metric space for classification given finite training samples. We define a discriminant adjacent matrix in favor of classification task, i.e., neighboring samples in the same class are squeezed but those in different classes are separated as far as possible. The optimal dimensionality of the metric space can be estimated by spectral analysis in the proposed method, which is of great significance for high-dimensional patterns. Experiments with various datasets demonstrate the effectiveness of our method.

References

[1]

Bar-Hillel, A., Hertz, T., Shental, N., & Weinshall, D. (2003). Learning distance functions using equivalence relations. Proceedings of the 20th International Conference on Machine Learning.

Digital Library

[2]

Belhumeur, P. N., Hespanda, J., & Kiregeman, D. (1997). Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711--720.

Digital Library

[3]

Belkin, M., & Niyogi, P. (2001). Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in Neural Information Processing Systems. Cambridge, MA.

[4]

Chen, H.-T., Chang, H.-W., & Liu, T.-L. (2005). Local discriminant embedding and its variants. IEEE Conference on Computer Vision and Pattern Recognition.

Digital Library

[5]

Cheng, J., Liu, Q., Lu, H., & Chen, Y.-W. (2004). A supervised nonlinear local embedding for face recognition. International Conference on Image Processing.

[6]

Fukunaga, K. (1990). Introduction to statistical pattern recognition. Boston: Academic Press. 2nd edition, 441--443, 466--467.

Digital Library

[7]

Goldberger, J., Roweis, S., Hinton, G., & Salakhutdinov, R. (2004). Neighborhood components analysis. Advances in Neural Information Processing Systems. Cambridge, MA.

[8]

Graham, D. B., & Allinson, N. M. (1998). Characterizing virtual eigensignatures for general purpose face recognition. Face Recognition: From Theory to Application, 163.

[9]

Hastie, T., & Tibshirani, R. (1996). Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(6), 607--616.

Digital Library

[10]

He, X., Yan, S., Hu, Y., Niyogi, P., & Zhang, H. (2005). Face recognition using laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3), 328--340.

Digital Library

[11]

Murphy, P. M., & Aha, D. W. (1994). UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html.

[12]

Qiu, X., & Wu, L. (2005). Stepwise nearest neighbor discriminant analysis. Proceedings of the 19th International Joint Conference on Artificial Intelligence.

Digital Library

[13]

Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323--2326.

[14]

Sugiyama, M. (2006). Local fisher discriminant analysis for supervised dimensionality reduction. Proceedings of the 23rd International Conference on Machine Learning. Pittsburgh, PA.

Digital Library

[15]

Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290, 2319--2323.

[16]

Wang, G., & Shi, R. (1988). Theory of matrix. Beijing, China: Defense Industry Press, 376--378 (in Chinese).

[17]

Weinberger, K. Q., Blitzer, J., & Saul, L. K. (2005). Distance metric learning for large margin nearest neighbor classification. Advances in Neural Information Processing Systems. Cambridge, MA.

[18]

Xing, E. P., Ng, A. Y., Jordan, M. I., & Russell, S. (2002). Distance metric learning, with application to clustering with side-information. Advances in Neural Information Processing Systems. Cambridge, MA.

[19]

Yan, S., Xu, D., Zhang, B., & Zhang, H.-J. (2005). Graph embedding: A general framework for dimensionality reduction. IEEE Conference on Computer Vision and Pattern Recognition.

Digital Library

Cited By

Luo YWong YKankanhalli MZhao Q(2020)$\mathcal{G}$ -Softmax: Improving Intraclass Compactness and Interclass Separability of FeaturesIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2019.290973731:2(685-699)Online publication date: Feb-2020
https://doi.org/10.1109/TNNLS.2019.2909737
Griebel MRieger CZwicknagl B(2018)Regularized Kernel-Based Reconstruction in Generalized Besov SpacesFoundations of Computational Mathematics10.1007/s10208-017-9346-z18:2(459-508)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1007/s10208-017-9346-z
Ye HZhan DSi XJiang Y(2017)Learning mahalanobis distance metricProceedings of the 26th International Joint Conference on Artificial Intelligence10.5555/3172077.3172352(3315-3321)Online publication date: 19-Aug-2017
https://dl.acm.org/doi/10.5555/3172077.3172352
Show More Cited By

Optimal dimensionality of metric space for classification
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning

Recommendations

A visualization metric for dimensionality reduction

Data visualization of high-dimensional data is possible through the use of dimensionality reduction techniques. However, in deciding which dimensionality reduction techniques to use in practice, quantitative metrics are necessary for evaluating the ...
Dimensionality reduction for classification
Robust dimensionality reduction via feature space to feature space distance metric learning
Abstract
Images are often represented as vectors with high dimensions when involved in classification. As a result, dimensionality reduction methods have to be developed to avoid the curse of dimensionality. Among them, Laplacian eigenmaps (LE) have ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '07: Proceedings of the 24th international conference on Machine learning

June 2007

1233 pages

ISBN:9781595937933

DOI:10.1145/1273496

Editor:
Zoubin Ghahramani
University of Cambridge, United Kingdom

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Machine Learning Journal

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ICML '07 & ILP '07

Sponsor:

ICML '07 & ILP '07: The 24th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 20 - 24, 2007

Oregon, Corvalis, USA

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
322
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Luo YWong YKankanhalli MZhao Q(2020)$\mathcal{G}$ -Softmax: Improving Intraclass Compactness and Interclass Separability of FeaturesIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2019.290973731:2(685-699)Online publication date: Feb-2020
https://doi.org/10.1109/TNNLS.2019.2909737
Griebel MRieger CZwicknagl B(2018)Regularized Kernel-Based Reconstruction in Generalized Besov SpacesFoundations of Computational Mathematics10.1007/s10208-017-9346-z18:2(459-508)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1007/s10208-017-9346-z
Ye HZhan DSi XJiang Y(2017)Learning mahalanobis distance metricProceedings of the 26th International Joint Conference on Artificial Intelligence10.5555/3172077.3172352(3315-3321)Online publication date: 19-Aug-2017
https://dl.acm.org/doi/10.5555/3172077.3172352
Shao GSang N(2017)Regularized max-min linear discriminant analysisPattern Recognition10.1016/j.patcog.2016.12.03066:C(353-363)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1016/j.patcog.2016.12.030
Ye HZhan DJiang Y(2016)Instance specific metric subspace learningProceedings of the Thirtieth AAAI Conference on Artificial Intelligence10.5555/3016100.3016216(2272-2278)Online publication date: 12-Feb-2016
https://dl.acm.org/doi/10.5555/3016100.3016216
Phienthrakul TSamnienggam M(2016)Comparative Results of Attribute Reduction Techniques for Thai Handwritten Recognition with Support Vector MachinesRecent Advances in Information and Communication Technology 201610.1007/978-3-319-40415-8_8(67-77)Online publication date: 12-Jun-2016
https://doi.org/10.1007/978-3-319-40415-8_8
Chi YGriffith EGoulermas JRalph J(2015)Binary Data Embedding Framework for Multiclass ClassificationIEEE Transactions on Human-Machine Systems10.1109/THMS.2015.240491345:4(453-464)Online publication date: Aug-2015
https://doi.org/10.1109/THMS.2015.2404913
Peng JSeetharaman GFan WVarde A(2013)Exploiting fisher and fukunaga-koontz transforms in chernoff dimensionality reductionACM Transactions on Knowledge Discovery from Data10.1145/2499907.24999117:2(1-25)Online publication date: 2-Aug-2013
https://dl.acm.org/doi/10.1145/2499907.2499911
Rodriguez-Martinez ETingting Mu Jianmin Jiang Goulermas J(2013)Automated Induction of Heterogeneous Proximity Measures for Supervised Spectral EmbeddingIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2013.226161324:10(1575-1587)Online publication date: Oct-2013
https://doi.org/10.1109/TNNLS.2013.2261613
Zhou XGan PYan CLi G(2012)Towards the Optimal Discriminant SubspaceProceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 0110.5555/2457524.2457680(181-187)Online publication date: 4-Dec-2012
https://dl.acm.org/doi/10.5555/2457524.2457680
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten