Text Categorization Using Hybrid Multiple Model Schemes

Abstract

Automatic text categorization techniques using inductive machine learning methods have been employed in various applications. In this paper, we review the characteristics of the existing multiple model schemes which include bagging, boosting, and stacking. Multiple model schemes try to optimize the predictive accuracy by combining predictions of diverse models derived from different versions of training examples or learning algorithms. In this study, we develop hybrid schemes which combine the techniques of existing multiple model schemes to improve the accuracy of text categorization, and conduct experiments to evaluate the performances of the proposed schemes on MEDLINE, Usenet news, and Web document collections. The experiments demonstrate the effectiveness of the hybrid multiple model schemes. Boosted stacking algorithms, that are a kind of the extended stacking algorithms proposed in this study, yield higher accuracies relative to the conventional multiple model schemes and single model schemes.