Abstract
The performance of similarity-based classifiers, such as K-nn, depends highly on the input space representation, both regarding feature relevence and feature interdependence. Feature weighting is a known technique aiming at improving performance by adjusting the importance of each feature at the classification decision. In this paper, we propose a non-linear feature transform for continuous features, which we call feade. The transform is applied prior to classification providing a new set of features, each one resulting by deforming in a local base the original feature according to a generalised mutual information metric for different regions of the feature value range. The algorithm is particularly efficient because it requires linear complexity in respect to the dimensions and the sample and does not need other classifier pre-training. Evaluation on real datasets shows an improvement in the performance of the K-nn classifier.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aha, D.W.: Feature weighting for lazy learning algorithms. In: Motoda, H.L.H. (ed.) Feature Extraction, Construction and Selection: A Data Mining Perspective, Kluwer, Norwell (1998)
Toussaint, G.: Proximity graphs for neares neighbor decision rules: recent progress. In: Proceedings of INTERFACE-2002, 34th Symposium on Computing ans Statistics (2002)
Tsallis, C.: Entropic nonextensivity: A possible measure of complexity. Technical Report 00-08-043, Santa-Fe Institute (2000)
Cover, T.M., Thomas, J.A.: Elements of Information Theory. John Wiley & Sons, Inc., Chichester (1991)
Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press Limited, London (1990)
Wilson, D., Martinez, R., Improved, T.: heterogeneous distance functions. Journal of Artificial Intelligence Research (1997)
Brewer, J.: A bayesian model for local smoothinh in kernel density estimation. Statistics and Computing 10, 299–309 (2000)
Abramson, I.: On bandwidth variation in kernel estimates — a square root law. Annals of Statistics (1982)
Paninski, L.: Estimation of entropy and mutual information. Neural Computation 15, 1191–1253 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Petridis, S., Perantonis, S.J. (2004). Feature Deforming for Improved Similarity-Based Learning. In: Vouros, G.A., Panayiotopoulos, T. (eds) Methods and Applications of Artificial Intelligence. SETN 2004. Lecture Notes in Computer Science(), vol 3025. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24674-9_22
Download citation
DOI: https://doi.org/10.1007/978-3-540-24674-9_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21937-8
Online ISBN: 978-3-540-24674-9
eBook Packages: Springer Book Archive