KerMinSVM for imbalanced datasets with a case study on arabic comics classification

Abstract

Many studies have been performed to classify large-sized text documents using different classifiers, ranging from simple distance classifiers such as K-Nearest-Neighbor (KNN) to more advanced classifiers such as Support Vector Machines. Traditional approaches fail when a short text is encountered due to sparsity resulting from a limited number of words. Another common problem in text classification is class imbalance (CI). CI occurs when one class of the data contains most of the samples while the other class contains only a few. Standard classifiers, when applied to imbalanced data, result in high accuracy for the majority class and low accuracy for the minority one. We were motivated to propose a novel framework for classifying the content of Arabic comics; therefore, we propose KerMinSVM, a kernel extension of our previously proposed MinSVM coupled with a new dimensionality featuring a reduction scheme based on word root frequency ratios (WRFR). KerMinSVM was tested on multiple imbalanced benchmark datasets, and the results were verified using three measures: accuracy, F-measure, and statistical analysis. WRFR was applied to the manual construction of the Arabic comic text dataset to detect strong content in children's comic books. Test results revealed that our proposed framework outperformed most of the methods for imbalanced datasets and short text classification.

Introduction

Due to the rapid development and spread of the internet, different types of short texts have been produced, such as web search snippets, chat messages, comments, status updates, tweets, news feeds, books, movie synopses and reviews. Classifying short text is of great importance for several purposes and applications, such as filtering offensive comments or assessing the satisfaction of customers with a certain product. Another example of short text is found in comic books. This text is usually unstructured and takes the form of brief conversations, consisting of multiple short sentences. Since short texts tend to have a sparse feature vector and exhibit class imbalance (CI), they cannot be classified with good accuracy using standard techniques.

An imbalanced dataset is one in which the different classification categories are not equally represented. A class that comprises many samples is referred to as the ‘majority class’ and conversely a class that contains very few samples is known as the ‘minority class’. As stated before, when performing classification on an imbalanced dataset, the classifier tends to achieve a high level of accuracy for the majority class, but low accuracy for the minority ones. This is because most of the classification algorithms focus on maximizing overall accuracy, without taking into consideration the accuracy of each class. In imbalanced datasets, the impact of minority samples is more pronounced than the majority samples. Misclassifying these minority samples will inevitably result in misleading and inaccurate information and hence undermine the aims of the application (Awad and Khanna, 2015).

Comics are usually popular amongst children. However, a number of these comics include strong content, such as conflicts, war, weaponry, and martyrdom, which are topics unsuitable for a younger audience. In general, the number of comics that contains strong material is very small with respect to the comics that are suitable for children. Nevertheless, detecting strong content in comic books is crucial. In this direction, we propose a new framework to classify short Arabic texts. Taking into consideration the root base nature of the Arabic language, we reduce the sparsity and dimensionality of the feature vector without adding external information to the original data. This methodology allows the roots of words to be used as features. It groups words of same root in one feature, and consequently reduces data dimensionality. To reduce the feature vector length even further, roots are grouped together based on their semantic similarities. To test the feature reduction technique, a dataset of Arabic comic text is manually constructed and annotated. Grouping similar roots together gave better representation of the constructed dataset and reduced the sparsity of the feature vector. To improve the classification accuracy of Support Vector Machines (SVM) on imbalanced data, a kernel extension to the Minority Support Vector Machine (MinSVM) classifier we proposed earlier (Ajeeb et al., 2013) has been developed and tested.

The remainder of this paper includes, under Section II, the literature review for the techniques used to improve imbalanced data classification as well as previous work on Arabic text and on short text classifications. Section III presents the proposed framework containing KerMinSVM for imbalanced data classification and WRFR, the new feature extraction approach for short Arabic text. Experimental results are presented in Section IV, followed by concluding remarks in Section V.