Feature selection methods for text classification: a systematic literature review

Abstract

Feature Selection (FS) methods alleviate key problems in classification procedures as they are used to improve classification accuracy, reduce data dimensionality, and remove irrelevant data. FS methods have received a great deal of attention from the text classification community. However, only a few literature surveys include them focusing on text classification, and the ones available are either a superficial analysis or present a very small set of work in the subject. For this reason, we conducted a Systematic Literature Review (SLR) that asses 1376 unique papers from journals and conferences published in the past eight years (2013–2020). After abstract screening and full-text eligibility analysis, 175 studies were included in our SLR. Our contribution is twofold. We have considered several aspects of each proposed method and mapped them into a new categorization schema. Additionally, we mapped the main characteristics of the experiments, identifying which datasets, languages, machine learning algorithms, and validation methods have been used to evaluate new and existing techniques. By following the SLR protocol, we allow the replication of our revision process and minimize the chances of bias while classifying the included studies. By mapping issues and experiment settings, our SLR helps researchers to develop and position new studies with respect to the existing literature.

Appendix A. List of acronyms

ACC:

Accuracy Measure

ACC2:

Balanced Accuracy Measure

ALOFT:

At Least One FeaTure

ANOVA:

Analysis of Variance

ACA:

Bit-priori Association Classification Algorithm

BBHA:

Binary Black Hole Algorithm

BFSM:

Blended Feature Selection Method

BGSA:

Binary Gravitational Search Algorithm

BMI:

Balanced Mutual Information

BoDW:

Bag of Discriminative Words

BoW:

Bag of Words

BPSO:

Binary Particle Swarm Optimization

CAS:

Correlative Association Score

CDM:

Class Discriminating Measure

CHI:

Chi-square

CMFS:

Comprehensively Measure Feature Selection

CNN:

Convolutional Neural Network

CrowdFS:

Crowd-based Feature Selection

CSO:

Cat Swarm Optimization

DBN:

Deep Belief Network

DF:

Document Frequency

DFS:

Discriminative Features Selection

DFS$^{*}$:

Distinguishing Feature Selector

DGBFS:

Diversified Greedy Backward-Forward Search

DPP:

Discriminative Personal Purity

DT:

Decision Tree

EEFS:

Ensemble Embedded Feature Selection

FRFS:

Fuzzy Rough Feature Selection

FS:

Feature Selection

GAWA:

Genetic Algorithm and Wrapper Approaches

GFSS:

Global Filter-based Feature Selection Scheme

GI:

Gini Index

GPSO:

Geometric Particle Swarm Optimization

HAN:

Hierarchical Attention Network

HRFS:

Hebb Rule Based Feature Selection

IDF:

Inverse Document Frequency

IG:

Information Gain

IPSO:

Improved Particle Swarm Optimization

ISCA:

Improved Sine Cosine Algorithm

KNN:

k-Nearest Neighbors

LDA:

Latent Dirichlet Allocation

LSAN:

Latent Selection Augmented Naive Bayes

MBF:

Markov Blanket Filter

MFS:

Meta Feature Selection

MFSLFD:

Memetic Feature Selection based on Label Frequency Difference

MI:

Mutual Information

MMI:

Multivariate Mutual Information

MMR:

Max-Min Ratio

MOANOFS:

Multi-Objective Automated Negotiation based Online Feature Selection

MORDC:

Multi-Objective Relative Discriminative Criterion

MRDC:

Multivariate Relative Discrimination Criterion

NB:

Naive Bayes

NDM:

Normalized Difference Measure

OR:

Odds Ratio

OS-FS:

Optimized Swarm Search-based Feature Selection

PCT:

Pairwise Comparison Transformation

POS:

Part of Speech

POSFilter:

Part of Speech Filter

PSO:

Particle Swarm Optimization

RCV1:

Reuters Corpus Volume I

RDC:

Relative Discrimination Criterion

RF:

Random Forest

RFE:

Recursive Feature Elimination

RP-GSO:

Random Projection and Gram-Schmidt Orthogonalization

SAIG:

Sparsity Adjusted Information Gain

SBATFS:

Spark BAT Feature Selection

SIGCHI:

Square of Information Gain and Chi-square

SLR:

Systematic Literature Review

SMOTE:

Synthetic Minority Oversampling Technique

SVM:

Support Vector Machines

SVM-RFE:

Support Vector Machine-Recursive Feature Elimination

SWA:

Small World Algorithm

t-Test:

Student’s t-Test

TF :

Term Frequency

TF-IDF:

Term Frequency-Inverse Document Frequency

WFSAIG:

Wrapper Feature Selection Algorithm based on Iterated Greedy

WI-OMFS:

Wolf Intelligence Based Optimization of Multi-Dimensional Feature Selection Approach