skip to main content
10.1145/1008992.1009034acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
Article

Feature selection using linear classifier weights: interaction with classification models

Published: 25 July 2004 Publication History

Abstract

This paper explores feature scoring and selection based on weights from linear classification models. It investigates how these methods combine with various learning models. Our comparative analysis includes three learning algorithms: Naïve Bayes, Perceptron, and Support Vector Machines (SVM) in combination with three feature weighting methods: Odds Ratio, Information Gain, and weights from linear models, the linear SVM and Perceptron. Experiments show that feature selection using weights from linear SVMs yields better classification performance than other feature weighting methods when combined with the three explored learning algorithms. The results support the conjecture that it is the sophistication of the feature weighting method rather than its apparent compatibility with the learning algorithm that improves classification performance.

References

[1]
Janez Brank, Marko Grobelnik, Nataša Milić-Frayling, and Dunja Mladenić. Feature selection using support vector machines. Proc. of the 3rd Int. Conf. on Data Mining Methods and Databases for Engineering, Finance, and Other Fields, Bologna, Italy, September 2002.
[2]
Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning, 20(3):273--297, September 1995.
[3]
Thorsten Joachims. (1999). Making large-scale support vector machine learning practical. In B. Schölkopf et al. (Eds.), Advances in kernel methods: Support vector learning. MIT Press, 1999, pp. 169--184.
[4]
Werner Krauth and Marc Mézard. Learning algorithms with optimal stability in neural networks. Jour. Physics A 20, L745-L752, August 1987.
[5]
Andrew McCallum and Kamal Nigam. A comparison of event models for Naïve Bayes text categorization. AAAI Workshop on Learning for Text Categorization (pp. 41--48). AAAI Press, 1998.
[6]
Dunja Mladenić and Marko Grobelnik. Feature selection for unbalanced class distribution and Naïve Bayes. Proc. 16th Int. Conf. on Mach. Learning. San Francisco: Morgan Kaufmann, pp. 258--267, 1999.
[7]
J. Ross Quinlan. Constructing decision trees. In: C4.5: Programs for machine learning, pp. 17--26. Morgan Kaufmann, 1993.
[8]
Frank Rosenblatt. The Perceptron: A probabilistic model for information storage and organization in the brain. Psych. Review 65(6), 386--408. Reprinted in: J. A. D. Anderson, E. Rosenfeld (Eds.), Neurocomputing: foundations of research. Cambridge, MA: MIT Press, 1988.
[9]
Vikas Sindhwani, Pushpak Bhattacharya, and Subrata Rakshit. Information theoretic feature crediting in multiclass Support Vector Machines. 1st SIAM Int. Conf. on Data Mining (SDM 2001), Chicago, IL, USA, April 5-7, 2001. SIAM, 2001.
[10]
Lawrence Shih, Yu-Han Chang, Jason Rennie, David Karger. Not too hot, not too cold: The Bundled-SVM is just right! Workshop on Text Learning (TextML-2002), ICML, Sydney, Australia, July 8, 2002.
[11]
Soumen Chakrabarti, Shourya Roy, Mahesh V. Soundalgekar: Fast and accurate text classification via multiple linear discriminant projections. Proceedings of the 28th International Conference on Very Large Data Bases (VLDB 2002), Hong Kong, China, August 20--23, 2002, pp. 658--669.

Cited By

View all
  • (2025)An improved soft voting-based machine learning technique to detect breast cancer utilizing effective feature selection and SMOTE-ENN class balancingDiscover Artificial Intelligence10.1007/s44163-025-00224-w5:1Online publication date: 20-Jan-2025
  • (2024)Research on Text Classification Methods by Fusing DeBERTa Model with Graph Convolutional NetworksArtificial Intelligence and Robotics Research10.12677/airr.2024.13407213:04(715-725)Online publication date: 2024
  • (2024)Using Artificial Intelligence for Rheumatic Heart Disease Detection by Echocardiography: Focus on Mitral RegurgitationJournal of the American Heart Association10.1161/JAHA.123.03125713:2Online publication date: 16-Jan-2024
  • Show More Cited By

Index Terms

  1. Feature selection using linear classifier weights: interaction with classification models

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '04: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
    July 2004
    624 pages
    ISBN:1581138814
    DOI:10.1145/1008992
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 July 2004

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. SVM normal
    2. feature scoring
    3. feature selection
    4. information retrieval
    5. linear SVM
    6. text classification
    7. vector representation

    Qualifiers

    • Article

    Conference

    SIGIR04
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)55
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)An improved soft voting-based machine learning technique to detect breast cancer utilizing effective feature selection and SMOTE-ENN class balancingDiscover Artificial Intelligence10.1007/s44163-025-00224-w5:1Online publication date: 20-Jan-2025
    • (2024)Research on Text Classification Methods by Fusing DeBERTa Model with Graph Convolutional NetworksArtificial Intelligence and Robotics Research10.12677/airr.2024.13407213:04(715-725)Online publication date: 2024
    • (2024)Using Artificial Intelligence for Rheumatic Heart Disease Detection by Echocardiography: Focus on Mitral RegurgitationJournal of the American Heart Association10.1161/JAHA.123.03125713:2Online publication date: 16-Jan-2024
    • (2024)Geoinference of author affiliations using NLP-based text classificationScientific Reports10.1038/s41598-024-73318-714:1Online publication date: 16-Oct-2024
    • (2024)Abstract Interpretation-Based Feature Importance for Support Vector MachinesVerification, Model Checking, and Abstract Interpretation10.1007/978-3-031-50524-9_2(27-49)Online publication date: 15-Jan-2024
    • (2023)Uncovering the Gut–Liver Axis Biomarkers for Predicting Metabolic Burden in MiceNutrients10.3390/nu1515340615:15(3406)Online publication date: 31-Jul-2023
    • (2023)Beyond Face Value: Identifying Irony and Sarcasm in Short Texts Through Multiclass Classification Techniques2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN)10.1109/ICPCSN58827.2023.00018(74-79)Online publication date: Jun-2023
    • (2022)Machine Learning Data Analysis Highlights the Role of Parasutterella and Alloprevotella in Autism Spectrum DisordersBiomedicines10.3390/biomedicines1008202810:8(2028)Online publication date: 19-Aug-2022
    • (2022)A Novel Hybrid Model Combining the Support Vector Machine (SVM) and Boosted Regression Trees (BRT) Technique in Predicting PM10 ConcentrationAtmosphere10.3390/atmos1312204613:12(2046)Online publication date: 7-Dec-2022
    • (2022)Privacy-Preserving and Publicly Verifiable Matrix MultiplicationIEEE Transactions on Services Computing10.1109/TSC.2022.3215499(1-13)Online publication date: 2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media