Abstract
We present a performance analysis of three linear dimensionality reduction techniques: Fisher’s discriminant analysis (FDA), and two methods introduced recently based on the Chernoff distance between two distributions, the Loog and Duin (LD) method, which aims to maximize a criterion derived from the Chernoff distance in the original space, and the one introduced by Rueda and Herrera (RH), which aims to maximize the Chernoff distance in the transformed space. A comprehensive performance analysis of these methods combined with two well-known classifiers, linear and quadratic, on synthetic and real-life data shows that LD and RH outperform FDA, specially in the quadratic classifier, which is strongly related to the Chernoff distance in the transformed space. In the case of the linear classifier, the superiority of RH over the other two methods is also demonstrated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aladjem, M.: Linear Discriminant Analysis for Two Classes Via Removal of Classification Structure. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(2), 187–192 (1997)
Cooke, T.: Two Variations on Fisher’s Linear Discriminant for Pattern Recognition. IEEE Transations on Pattern Analysis and Machine Intelligence 24(2), 268–273 (2002)
Du, Q., Chang, C.: A Linear Constrained Distance-based Discriminant Analysis for Hyperspectral Image Classification. Pattern Recognition 34(2), 361–373 (2001)
Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. John Wiley and Sons, Inc., New York (2000)
Lippman, R.: An Introduction to Computing with Neural Nets. In: Neural Networks: Theoretical Foundations and Analsyis, pp. 5–24. IEEE Press, Los Alamitos (1992)
Loog, M., Duin, P.W.: Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA: The Chernoff Criterion. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(6), 732–739 (2004)
Loog, M., Duin, R.: Non-iterative Heteroscedastic Linear Dimension Reduction for Two-Class Data. In: Caelli, T.M., Amin, A., Duin, R.P.W., Kamel, M.S., de Ridder, D. (eds.) SPR 2002 and SSPR 2002. LNCS, vol. 2396, pp. 508–517. Springer, Heidelberg (2002)
Lotlikar, R., Kothari, R.: Adaptive Linear Dimensionality Reduction for Classification. Pattern Recognition 33(2), 185–194 (2000)
Murphy, O.: Nearest Neighbor Pattern Classification Perceptrons. In: Neural Networks: Theoretical Foundations and Analysis, pp. 263–266. IEEE Press, Los Alamitos (1992)
Newman, D., Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases, University of California, Irvine, Dept. of Computer Science (1998)
Rao, A., Miller, D., Rose, K., Gersho, A.: A Deterministic Annealing Approach for Parsimonious Design of Piecewise Regression Models. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(2), 159–173 (1999)
Raudys, S.: On Dimensionality, Sample Size, and Classification Error of Nonparametric Linear Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(6), 667–671 (1997)
Raudys, S.: Evolution and Generalization of a Single Neurone: I. Single-layer Perception as Seven Statistical Classifiers. Neural Networks 11(2), 283–296 (1998)
Raudys, S.: Evolution and Generalization of a Single Neurone: II. Complexity of Statistical Classifiers and Sample Size Considerations. Neural Networks 11(2), 297–313 (1998)
Rueda, L.: Selecting the Best Hyperplane in the Framework of Optimal Pairwise Linear Classifiers. Pattern Recognition Letters 25(2), 49–62 (2004)
Rueda, L., Herrera, M.: Linear Discriminant Analysis by Maximizing the Chernoff Distance in the Transformed Space (submitted for Publication) (2006)
Rueda, L., Oommen, B.J.: On Optimal Pairwise Linear Classifiers for Normal Distributions: The Two-Dimensional Case. IEEE Transations on Pattern Analysis and Machine Intelligence 24(2), 274–280 (2002)
Webb, A.: Statistical Pattern Recognition, 2nd edn. John Wiley & Sons, New York (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ali, M.L., Rueda, L., Herrera, M. (2006). On the Performance of Chernoff-Distance-Based Linear Dimensionality Reduction Techniques. In: Lamontagne, L., Marchand, M. (eds) Advances in Artificial Intelligence. Canadian AI 2006. Lecture Notes in Computer Science(), vol 4013. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11766247_40
Download citation
DOI: https://doi.org/10.1007/11766247_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34628-9
Online ISBN: 978-3-540-34630-2
eBook Packages: Computer ScienceComputer Science (R0)