Abstract
Dimension reduction methods are often applied in machine learning and data mining problems. Linear subspace methods are the commonly used ones, such as principal component analysis (PCA), Fisher’s linear discriminant analysis (FDA), et al. In this paper, we describe a novel feature extraction method for binary classification problems. Instead of finding linear subspaces, our method finds lower- dimensional affine subspaces for data observations. Our method can be understood as a generalization of the Fukunaga-Koontz Transformation. We show that the proposed method has a closed-form solution and thus can be solved very efficiently. Also we investigate the information-theoretical properties of the new method and study the relationship of our method with other methods. The experimental results show that our method, as PCA and FDA, can be used as another preliminary data-exploring tool to help solve machine learning and data mining problems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Barker, M., Rayens, W.: Partial least squares for discrimination. Journal of Chemometrics 17(3), 166–173 (2003)
Bartlett, M.S.: Further aspects of the theory of multiple regression. Proc. Camb. Phil. Soc. 34, 33–40 (1938)
Belhumeur, P.N., Hespanha, J., Kriegman, D.J.: Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 711–720 (1997)
Dai, J.J, Lieu, L., Rocke, D.: Dimension reduction for classification with gene expression microarray data. Stat Appl Genet Mol Biol, 5 Article6 (2006)
Dornhege, G., Blankertz, B., Curio, G., Müller, K.-R.: Increase information transfer rates in BCI by CSP extension to multi-class. In: Advances in Neural Information Processing Systems 16 (2004)
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1999)
Foley, D.H., Sammon Jr., J.W.: An optimal set of discriminant vectors. IEEE Transactions on Computers 24, 281–289 (1975)
Fukunaga, K.: Introduction to statistical pattern recognition, 2nd edn. Academic Press, London (1990)
Fukunaga, K., Koontz, W.: Application of the Karhunen-Loéve expansion to feature selection and ordering. IEEE Transactions on Computers 19, 311–318 (1970)
Golub, G.H., Van Loan, C.F.: Matrix Computations. The Johns Hopkins University Press, Baltimore (1996)
Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning. Springer, Heidelberg (2001)
Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1990)
Huo, X., Elad, M., Flesia, A.G., Muise, B., Stanfill, R., Friedman, J., Popescu, B., Chen, J., Mahalanobis, A., Donoho, D.L.: Optimal reduced-rank quadratic classifiers using the Fukunaga-Koontz transform, with applications to automated target recognition. In: SPIE’s 7th International Symposium on Aerospace/Defense Sensing
De la Torre, F., Kanade, T.: Multimodal oriented discriminant analysis. In: ICML 2005. Proceedings of the 22nd international conference on Machine learning, New York, NY, USA, pp. 177–184. ACM Press, New York (2005)
Mika, S., Ratsch, G., Weston, J., Scholkopf, B., Muller, K.: Fisher discriminant analysis with kernels. In: Proceedings of IEEE Neural Networks for Signal Processing Workshop 1999 (1999)
Ramoser, H., Müller-Gerking, J., Pfurtscheller, G.: Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans. Rehab. Eng. 8, 441–446 (2000)
Rätsch, G., Onoda, T., Müller, K.-R.: Soft margins for adaboost. Mach. Learn. 42(3), 287–320 (2001)
Rosipal, R., Krämer, N.: Overview and recent advances in partial least squares. In: Saunders, C., Grobelnik, M., Gunn, S., Shawe-Taylor, J. (eds.) SLSFS 2005. LNCS, vol. 3940, pp. 34–51. Springer, Heidelberg (2005)
Rosipal, R., Trejo, L.J., Matthews, B.: Kernel PLS-SVM for linear and nonlinear classification. In: Fawcett, T., Mishra, N. (eds.) ICML, pp. 640–647. AAAI Press, Stanford (2003)
Yang, M.-H., Kriegman, D.J., Ahuja, N.: Detecting faces in images: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 24(1), 34–58 (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cao, W., Haralick, R. (2007). Affine Feature Extraction: A Generalization of the Fukunaga-Koontz Transformation. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2007. Lecture Notes in Computer Science(), vol 4571. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73499-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-540-73499-4_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73498-7
Online ISBN: 978-3-540-73499-4
eBook Packages: Computer ScienceComputer Science (R0)