Abstract
Feature selection (FS) plays a crucial role in machine learning to build a robust model for either learning or classification from a large amount of data. Among feature selection techniques, the Relief algorithm is one of the most common due to its simplicity and effectiveness. The performance of the Relief algorithm, however, could be dramatically affected by the consistency of the data patterns. For instance, Relief-F could become less accurate in the presence of noise. The accuracy would decrease further if an outlier sample was included in the dataset. Therefore, it is very important to select the samples to be included in the dataset carefully. This paper presents an effort to improve the effectiveness of Relief algorithm by filtering samples before selecting features. This method is termed Sample Filtering Relief Algorithm (SFRA). The main idea of this method is to discriminate outlier samples out of the main pattern using self organizing map (SOM) and then proceed with feature selection using the Relief algorithm. We have tested SFRA with a gene expression dataset of interferon-α(IFN-α) response of Hepatitis B patients that contains outlier data. SFRA could successfully remove outlier samples that have been verified by visual inspection by experts. Also, it has better accuracy in separating the relevant and irrelevant features than other feature selection methods considered.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Liu, H., Motoda, H.: Computational Methods of Feature Selection. Chapman & Hall/CRC, USA (2008)
Raman, B., Ioerger, T.R.: Instance Based Filter for Feature Selection. Journal of Machine Learning Research 1, 1–23 (2002)
Liu, B., Cui, Q., Jiang, T., Ma, S.: A Combinational Feature Selection and Ensemble Neural Network Method for Classification of Gene Expression data. BMC Bioinformatics 5 (2004)
Cho, S.B., Won, H.H.: Cancer Classification Using Ensemble of Neural Networks with Multiple Significant Gene Subsets. Appl. Intell. 26, 243–250 (2007)
Liu, H., Li, J., Wong, L.: A Comparative Study on Feature Selection and Classification Methods Using Gene Expression Profiles and Proteomic Patterns. Genome Informatics 13, 51–60 (2002)
Li, T., Zhang, C., Ogihara, M.: A Comparative Study of Feature Selection and Multiclass Classification Methods for Tissue Classification Based on Gene Expression. Bioinformatics 20, 2429–2437 (2004)
Liu, C.C., Chen, W.S.E., Lin, C.C., Liu, H.C., Chen, H.Y., Yang, P.C., Chang, P.C., Chen, J.: Topology-based Cancer Classification and Related Pathway Mining Using Microarray Data. Nucleic Acids Res. 34, 4069–4080 (2006)
Liu, J., Ranka, S., Kahveci, T.: Classification and Feature Selection Algorithms for Multi-class CGH data. Bioinformatics 24, i86–i95 (2008)
Liu, Y.: A Comparative Study on Feature Selection Methods for Drug Discovery. J. Chem. Inf. Comput. Sci. 44, 1823–1828 (2004)
Filzmoser, P., Maronna, R., Werner, M.: Outlier Identification in High Dimensions. Computational Statistics & Data Analysis 52, 1694–1711 (2008)
Lyons-Weiler, J., Patel, S., Becich, M.J., Godfrey, T.E.: Tests for Finding Complex Patterns of Differential Expression in Cancers: Towards Individualized medicine. BMC Bioinformatics 5 (2004)
Zeng, X.Q., Li, G.Z., Yang, J.Y., Yang, M.Q., Wu, G.F.: Dimension Reduction with Redundant Gene Elimination for Tumor Classification. BMC Bioinformatics 9 (2007)
Tomlins, S.A., Rhodes, D.R., Perner, S., Dhanasekaran, S.M., Mehra, R., Sun, X.W., Varambally, S., Cao, X., Tchinda, J., Kuefer, R., Lee, C., Montie, J.E., Shah, R.B., Pienta, K.J., Rubin, M.A., Chinnaiyan, A.M.: Recurrent Fusion of TMPRSS2 and ETS Transcription Factor Genes in Prostate Cancer. Science 310, 644–648 (2005)
Tibshirani, R., Hastie, T.: Outlier Sums for Differential Gene Expression Analysis. Biostatistics 8, 2–8 (2006)
Sun, Y.: Iterative RELIEF for Feature Weighting: Algorithms, Theories, and Applications. IEEE Transactions on Pattern Analysis and Machine Intelligent 29, 1035–1051 (2007)
Robnik-Šikonja, M., Kononenko, I.: Theoretical and Empirical Analysis of ReliefF and RReliefF. Machine Learning 53, 23–69 (2003)
Park, H., Kwon, H.C.: Extended Relief Algorithms in Sample-Based Feature Filtering. In: Sixth International Conference on Advanced Language Processing and Web Information Technology, pp. 123–128 (2007)
Yin, L., Huang, C.H., Ni, J.: Clustering of Gene Expression Data: Performance and Similarity Analysis. BMC Bioinformatics 7 (2006)
Amjady, N., Daraeepour, A.: Day-ahead Electricity Price Forecasting Using the Relief Algorithm and Neural Networks, vol. 978, pp. 1–7. IEEE, Los Alamitos (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Saethang, T., Prom-on, S., Meechai, A., Chan, J.H. (2009). Sample Filtering Relief Algorithm: Robust Algorithm for Feature Selection. In: Köppen, M., Kasabov, N., Coghill, G. (eds) Advances in Neuro-Information Processing. ICONIP 2008. Lecture Notes in Computer Science, vol 5507. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03040-6_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-03040-6_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03039-0
Online ISBN: 978-3-642-03040-6
eBook Packages: Computer ScienceComputer Science (R0)