Article

Local similarity discriminant analysis

Authors:

Maya R. GuptaAuthors Info & Claims

ICML '07: Proceedings of the 24th international conference on Machine learning

Pages 137 - 144

https://doi.org/10.1145/1273496.1273514

Published: 20 June 2007 Publication History

Abstract

We propose a local, generative model for similarity-based classification. The method is applicable to the case that only pairwise similarities between samples are available. The classifier models the local class-conditional distribution using a maximum entropy estimate and empirical moment constraints. The resulting exponential class conditional-distributions are combined with class prior probabilities and misclassification costs to form the local similarity discriminant analysis (local SDA) classifier. We compare the performance of local SDA to a non-local version, to the local nearest centroid classifier, the nearest centroid classifier, k-NN, and to the recently-developed potential support vector machine (PSVM). Results show that local SDA is competitive with k-NN and the computationally-demanding PSVM while offering the advantages of a generative classifier.

References

[1]

Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Trans. on Pattern Analysis and Machine Intelligence, 24, 509--522.

Digital Library

[2]

Bicego, M., Murino, V., Pelillo, M., & Torsello, A. (2006). Special issue on similarity-based classification. Pattern Recognition, 39.

Digital Library

[3]

Cost, S., & Salzberg, S. (1993). A weighted nearest neighbor algorithm for learning with symbolic features. Machine Learning, 10, 57--78.

[4]

Cover, T., & Thomas, J. (1991). Elements of information theory. New York: John Wiley and Sons.

Digital Library

[5]

Devroye, L., Gyorfi, L., & Lugosi, G. (1996). A probabilistic theory of pattern recognition. New York: Springer-Verlag Inc.

[6]

Duin, R. P. W., Pekalska, E., & de Ridder, D. (1999). Relational discriminant analysis. Pattern Recognition Letters, 20, 1175--1181.

Digital Library

[7]

Friedlander, M. P., & Gupta, M. R. (2006). On minimizing distortion and relative entropy. IEEE Trans. on Information Theory, 52, 238--245.

Digital Library

[8]

Gati, I., & Tversky, A. (1984). Weighting common and distinctive features in perceptual and conceptual judgments. Cognitive Psychology, 341--370.

[9]

Goldfarb, L. (1985). A new approach to pattern recognition. Progress in Pattern Recognition, 2, 241--402.

[10]

Graepel, T., Herbrich, R., & Obermayer, K. (1999). Classification on pairwise proximity data. Advances in Neural Information Processing Systems 11, 438--444.

Digital Library

[11]

Gupta, M. R., Cazzanti, L., & Koppal, A. J. (2007). Maximum entropy generative models for similarity-based learning. IEEE Intl. Symp. on Information Theory, to appear.

[12]

Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning. New York: Springer-Verlag.

[13]

Hochreiter, S., Mozer, M. C., & Obermayer, K. (2003). Coulomb classifiers: Generalizing support vector machines via an analogy to electrostatic systems. Advances in Neural Information Processing Systems 15, 545--552.

[14]

Hochreiter, S., & Obermayer, K. (2006). Support vector machines for dyadic data. Neural Computation, 18, 1472--1510.

Digital Library

[15]

Hoffmann, T., & Buhmann, J. (1997). Pairwise data clustering by deterministic annealing. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19.

Digital Library

[16]

Jacobs, D. W., Weinshall, D., & Gdalyahu, Y. (2000). Classification with nonmetric distances: Image retrieval and class representation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22, 583--600.

Digital Library

[17]

Jaynes, E. T. (1982). On the rationale for maximum entropy methods. Proc. of the IEEE, 70, 939--952.

[18]

Jaynes, E. T. (2003). Probability theory: the logic of science. Cambridge University Press.

[19]

Mitani, Y., & Hamamoto, Y. (2000). Classifier design based on the use of nearest neighbor samples. Proc. of the Intl. Conf. on Pattern Recognition, 769--772.

[20]

Mitani, Y., & Hamamoto, Y. (2006). A local mean-based nonparametric classifier. Pattern Recognition Letters, 27, 1151--1159.

Digital Library

[21]

Newman, D. J., Hettich, S., Blake, C. L., & Merz, C. J. (1998). UCI repository of machine learning databases.

[22]

Pekalska, E., Paclíc, P., & Duin, R. P. W. (2001). A generalized kernel approach to dissimilarity-based classification. Journal of Machine Learning Research, 175--211.

Digital Library

[23]

Rosch, E. (1973). Natural categories. Cognitive Psychology, 328--350.

[24]

Simard, P., Cun, Y. L., & Denker, J. (1993). Efficient pattern recognition using a new transformation distance. Advances in Neural Information Processing Systems 5, 50--68.

Digital Library

[25]

Stanfill, C., & Waltz, D. (1986). Toward memory-based reasoning. Communications of the ACM, 29, 1213--1228.

Digital Library

[26]

Tversky, A. (1977). Features of similarity. Psychological Review, 327--352.

[27]

Tversky, A., & Gati, I. (1978). Studies of similarity. In E. Rosch and B. Lloyd (Eds.), Cognition and categorization. Hillsdale, N.J.: Earlbaum.

[28]

Van Campenhout, J., & Cover, T. (1981). Maximum entropy and conditional probability. IEEE Trans. on Information Theory, 27, 483--489.

Digital Library

[29]

Weinshall, D., Jacobs, D. W., & Gdalyahu, Y. (1999). Classification in non-metric spaces. Advances in Neural Information Processing Systems 11, 838--844.

Digital Library

[30]

Zhang, H., Berg, A. C., Maire, M., & Malik, J. (2006). SVM-KNN: discriminative nearest neighbor classification for visual category recognition. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, 2126--2136.

Digital Library

Cited By

Bingöl MŞafak SAkın Ö(2024)Comparing some plant communities in a region of Türkiye via fuzzy similarityInternational Journal of Mathematics and Computer in Engineering10.2478/ijmce-2024-0020Online publication date: 10-Jan-2024
https://doi.org/10.2478/ijmce-2024-0020
Singha ABhowmik MBhattacherjee D(2020)Akin-based Orthogonal Space (AOS): a subspace learning method for face recognitionMultimedia Tools and Applications10.1007/s11042-020-08892-9Online publication date: 11-May-2020
https://doi.org/10.1007/s11042-020-08892-9
Xiong HZhang JHuang YLeach KBarnes L(2017)DaehrACM Transactions on Intelligent Systems and Technology10.1145/30071958:3(1-21)Online publication date: 8-Feb-2017
https://dl.acm.org/doi/10.1145/3007195
Show More Cited By

Local similarity discriminant analysis
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Recommendations

Regularizing the Local Similarity Discriminant Analysis Classifier
ICMLA '09: Proceedings of the 2009 International Conference on Machine Learning and Applications

We investigate parameter-based and distribution-based approaches to regularizing the generative, similarity-based classifier called local similarity discriminant analysis classifier (local SDA). We argue that regularizing distributions rather than ...
Local Tangent Space Discriminant Analysis

We propose a novel supervised dimensionality reduction method named local tangent space discriminant analysis (TSD) which is capable of utilizing the geometrical information from tangent spaces. The proposed method aims to seek an embedding space where ...
Joint Global and Local Structure Discriminant Analysis

Linear discriminant analysis (LDA) only considers the global Euclidean geometrical structure of data for dimensionality reduction. However, previous works have demonstrated that the local geometrical structure is effective for dimensionality reduction. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '07: Proceedings of the 24th international conference on Machine learning

June 2007

1233 pages

ISBN:9781595937933

DOI:10.1145/1273496

Editor:
Zoubin Ghahramani
University of Cambridge, United Kingdom

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Machine Learning Journal

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ICML '07 & ILP '07

Sponsor:

ICML '07 & ILP '07: The 24th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 20 - 24, 2007

Oregon, Corvalis, USA

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
249
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bingöl MŞafak SAkın Ö(2024)Comparing some plant communities in a region of Türkiye via fuzzy similarityInternational Journal of Mathematics and Computer in Engineering10.2478/ijmce-2024-0020Online publication date: 10-Jan-2024
https://doi.org/10.2478/ijmce-2024-0020
Singha ABhowmik MBhattacherjee D(2020)Akin-based Orthogonal Space (AOS): a subspace learning method for face recognitionMultimedia Tools and Applications10.1007/s11042-020-08892-9Online publication date: 11-May-2020
https://doi.org/10.1007/s11042-020-08892-9
Xiong HZhang JHuang YLeach KBarnes L(2017)DaehrACM Transactions on Intelligent Systems and Technology10.1145/30071958:3(1-21)Online publication date: 8-Feb-2017
https://dl.acm.org/doi/10.1145/3007195
Qiao MLi J(2016)Distance-based mixture modeling for classification via hypothetical local mappingStatistical Analysis and Data Mining10.5555/3160268.31602729:1(43-57)Online publication date: 1-Feb-2016
https://dl.acm.org/doi/10.5555/3160268.3160272
Ionescu RPopescu MIonescu RPopescu M(2016)Learning based on SimilarityKnowledge Transfer between Computer Vision and Text Mining10.1007/978-3-319-30367-3_2(15-37)Online publication date: 26-Apr-2016
https://doi.org/10.1007/978-3-319-30367-3_2
Yuh-Jyh Hu Min-Che Yu Hsiang-An Wang Zih-Yun Ting (2015)A Similarity-Based Learning Algorithm Using Distance TransformationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2015.239110927:6(1452-1464)Online publication date: 28-Apr-2015
https://dl.acm.org/doi/10.1109/TKDE.2015.2391109
Qiao MLi J(2015)Distance‐based mixture modeling for classification via hypothetical local mappingStatistical Analysis and Data Mining: The ASA Data Science Journal10.1002/sam.112859:1(43-57)Online publication date: 14-Aug-2015
https://doi.org/10.1002/sam.11285
Liu QJing XHu RYao YYang J(2013)Similarity preserving analysis based on sparse representation for image feature extraction and classification2013 IEEE International Conference on Image Processing10.1109/ICIP.2013.6738620(3013-3016)Online publication date: Sep-2013
https://doi.org/10.1109/ICIP.2013.6738620
Pahalawatta KGreen R(2013)Particle Detection and Classification in Photoelectric Smoke Detectors Using Image Histogram Features2013 International Conference on Digital Image Computing: Techniques and Applications (DICTA)10.1109/DICTA.2013.6691509(1-8)Online publication date: Nov-2013
https://doi.org/10.1109/DICTA.2013.6691509
Xu JYang JLai Z(2013)K-local hyperplane distance nearest neighbor classifier oriented local discriminant analysisInformation Sciences: an International Journal10.1016/j.ins.2012.12.045232(11-26)Online publication date: 1-May-2013
https://dl.acm.org/doi/10.1016/j.ins.2012.12.045
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten