Abstract
This paper proposes a method of finding a discriminative linear transformation that enhances the data's degree of conformance to the compactness hypothesis and its inverse. The problem formulation relies on inter-observation distances only, which is shown to improve non-parametric and non-linear classifier performance on benchmark and real-world data sets. The proposed approach is suitable for both binary and multiple-category classification problems, and can be applied as a dimensionality reduction technique. In the latter case, the number of necessary discriminative dimensions can be determined exactly. The sought transformation is found as a solution to an optimization problem using iterative majorization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
ARKADEV, A. and BRAVERMAN, E. (1966): Computers and Patter Recognition. Thompson, Washington, D.C.
BLAKE, C. and MERZ, C. (1998), UCI Repository of machine learning databases.
BORG, I. and GROENEN, P. J. F. (1997): Modern Multidimensional Scaling. New York, Springer.
COX, T., FERRY, G. (1993): Discriminant analysis using nonmetric multidimensional scaling. Pattern Recognition, 26(1), 145–153.
CRISTIANINI, N. and SHAWE-TAYLOR, J. (2000): An introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, Cambridge.
DE LEEUW, J. (1977): Applications of convex analysis to multidimensional scaling. Recent Developments in Statistics, 133–145.
DUDA, R. O. and HART, P. E. (1973): Pattern Classification and Scene Analysis. John Wiley & Sons, New York.
FISHER, R. A. (1936): The Use of Multiple Measures in Taxonomic Problems. Ann. Eugenics, 7, 179–188.
FIX, E. and HODGES, J. (1951): Discriminatory analysis: Nonparametric discrimination: Consistency properties. Tech. Rep. 4, USAF School of Aviation Medicine.
HAGER, W. (2001): Minimizing quadratic over a sphere. SIAM Journal on Optimization, 12(1), 188–208.
HEISER, W. (1995): Convergent computation by iterative majorization: Theory and applications in multidimensional data analysis. Recent advances in descriptive multivariate analysis, 157–189.
HUBER, P. (1964): Robust estimation of a location parameter. Annals of Mathematical Statistics, 35, 73–101.
KOONTZ, W. and FUKUNAGA, K. (1972): A nonlinear feature extraction algorithm using distance information. IEEE Trans. Comput., 21(1), 56–63.
KOSINOV, S. (2003): Visual object recognition using distance-based discriminant analysis. Tech. Rep. 03.07, Computer Vision and Multimedia Laboratory, Computing Centre, University of Geneva, Rue Général Dufour, 24, CH-1211, Geneva, Switzerland.
KROGH, A. and HERTZ, J. A. (1992): A Simple Weight Decay Can Improve Generalization. In: J. E. Moody, S. J. Hanson and R. P. Lippmann (Eds.), Advances in Neural Information Processing Systems, vol. 4, 950–957. Morgan Kaufmann Publishers, Inc.
ROJAS, M., SANTOS, S. and Sorensen, D. (2000): A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem. SIAM Journal on Optimization, 11(3), 611–646.
VAN DEUN, K. and GROENEN, P. J. F. (2003): Majorization Algorithms for Inspecting Circles, Ellipses, Squares, Rectangles, and Rhombi. Tech. rep., Econometric Institute Report EI 2003-35.
WEBB, A. (1995): Multidimensional scaling by iterative majorization using radial basis functions. Pattern Recognition, 28(5), 753–759.
ZHOU, X. and HUANG, T. (2001): Small Sample Learning during Multimedia Retrieval using BiasMap. In: IEEE Computer Vision and Pattern Recognition (CVPR'01), Hawaii.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin · Heidelberg
About this paper
Cite this paper
Kosinov, S., Marchand-Maillet, S., Pun, T. (2005). Iterative Majorization Approach to the Distance-based Discriminant Analysis. In: Weihs, C., Gaul, W. (eds) Classification — the Ubiquitous Challenge. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-28084-7_17
Download citation
DOI: https://doi.org/10.1007/3-540-28084-7_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25677-9
Online ISBN: 978-3-540-28084-2
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)