Abstract
We propose a document classifier based on principal component analysis (PCA) and one-class support vector machine (OCSVM), where PCA helps achieve dimensionality reduction and OCSVM performs classification. Initially, PCA is invoked on the document-term matrix resulting in choosing the top few principal components. Later, OCSVM is trained on the records of the matrix corresponding to the negative class. Then, we tested the trained OCSVM with the records of the matrix corresponding to the positive class. The effectiveness of the proposed model is demonstrated on the popular datasets, viz., 20NG, malware, Syskill, & Webert, and customer feedbacks of a Bank. We observed that the hybrid yielded very high accuracies in all datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Joachims, T.: Text categorization with Support Vector Machines: Learning with many relevant features. In: Machine Learning: ECML 98, LNCS, Vol. 1398, pp. 137–142 (1998).
Dorre, J., Gerstl, P., Seiffert, R.: Text mining: Finding nuggets in mountains of textual data. In: KDD 99, San Diego, CA, USA, pp. 398–401 (1999).
Salton, G., and McGill, M. J.: Introduction to Modern Information Retrieval. McGraw-Hill, Inc. New York, NY, USA (1986).
Maron, M. E.: Automatic Indexing: An Experimental Inquiry, Journal of the ACM 8 (3), 404–417 (1961).
Masand, B., Linoff, G., Waltz, D.: Classifying news stories using memory based reasoning. In: 15th ACM International Conference on Research and Development in Information Retrieval (SIGIR 92), Copenhagen, Denmark, pp. 59–65 (1992).
Manevitz, L. M., Yousef, M.: One-Class SVMs for document classification. Journal of Machine Learning Research, 139–154 (2001).
Yu, H., Han, J., Chang, K. C-C.: PEBL: Positive Example Based Learning for web page classification using SVM. In: KDD 02, Edmonton, Alberta, Canada, pp. 239–248 (2002).
Vert, R., and Vert, J-P.: Consistency and Convergence Rates of One-Class SVMs and Related Algorithms. Journal of Machine Learning Research, 817–854 (2006).
Metsis, V., Androutsopolos, I., Paliouras, G.: Spam filtering with Naive Bayes - Which Naive Bayes?. In: 3rd Conference on Email and AntiSpam (CEAS 06), Mountain view, California, pp. 28–69 (2006).
Murthy, P. M., and Murthy, M. N.: Discriminative Feature Analysis and Selection for Document Classification. ICONIP, Part I, LNCS 7663, pp. 366–374 (2012).
Pandey, M., Ravi, V.: Text and data mining to detect phishing websites and spam emails. In: Swarm, Evolutionary, and Memetic Computing Conference (SEMCCO), Part II, LNCS 8298, pp. 559–573 (2013).
Jun, S., Park, S. S., Jang, D. S.: Document clustering method using dimension reduction and support vector clustering to overcome sparseness. Expert Systems with Applications 41 (7), pp. 3204–3212 (2014).
Sundarkumar, G. G., Ravi, V.: A novel hybrid undersampling method for mining unbalanced datasets in banking and insurance. Engineering Applications of Artificial intelligence 37, 368–377 (2015).
Denis, F., Gilleron, R., and Tommasi, M.: Text classification from positive and unlabeled examples. In: 9th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU), Annecy, France, pp. 1927–1934 (2002).
Lee, W. S., and Liu, B.: Learning with positive and unlabeled examples using weighted Logistic Regression. In: 12th ICML’ 03, Washington, DC, pp. 448–455 (2003).
Manevitz, L. M., Yousef, M.: Document classification on neural networks using only positive examples. In: 23rd ACM International Conference on Research and Development in Information Retrieval (SIGIR 00), Athens, Greece, pp. 304–306 (2000).
Elkan, C., and Noto, K.: Learning classifiers from only positive and unlabeled data. In: KDD 08, August 24–27, Las Vegas, Nevada, USA, pp. 213-220 (2008).
Anderson, T. W.: Asymptotic theory for principal component analysis. Annals of Mathematical Statistics 34 (1), 122–148 (1963).
Jolliffe, I.T.: Principal Component Analysis. Springer Verlag (1986).
Burges, C. J. C.: Dimension reduction: A guided tour. Foundations and trends in Machine Learning 2 (4), 275–365 (2009).
Ferre, L.: Selection of components in principal component analysis: A comparison of methods. Computational statistics and data analytics 19 (6), 669–682 (1995).
Lian, H.: On feature selection with principal component analysis for one-class SVM. Pattern Recognition Letters 33 (9), 1027–1031 (2012).
Chen, Y., Zhou, X., and Huang, T. S.: One-class SVM for learning in image retrieval. In: International Conference on Image Processing (ICIP), Thessaloniki, Greece, pp. 34–37 (2001).
Liu, C., Wang, G., and Ning, W., Lin, X., Li, L., Liu, Z.: Anomaly detection in surveillance video using motion direction statistics. In: 17th International Conference on Image Processing, Hong Kong, pp. 717–720 (2010).
Wan, C., Mita, A.: An automatic pipeline monitoring system based on PCA and SVM. International Journal of Mathematical, Computational, Natural and Physical Engineering 2 (9), 90–96 (2008).
Newsgroups, http://qwone.com/~jason/20Newsgroups.
Csmining group. http://www.csmining.org/index.php/malicious-software-datasets-.html.
Syskill & Webert web page ratings, https://archive.ics.uci.edu/ml/machine-learning-databases/SyskillWebert-mld/.
IBM SPSS, http://www-01.ibm.com/software/in/analytics/spss/products/data-collection/.
Rapid Miner, https://rapidminer.com (2013).
MATLAB, www.mathworks.com (2012).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Shravan Kumar, B., Ravi, V. (2017). Text Document Classification with PCA and One-Class SVM. In: Satapathy, S., Bhateja, V., Udgata, S., Pattnaik, P. (eds) Proceedings of the 5th International Conference on Frontiers in Intelligent Computing: Theory and Applications . Advances in Intelligent Systems and Computing, vol 515. Springer, Singapore. https://doi.org/10.1007/978-981-10-3153-3_11
Download citation
DOI: https://doi.org/10.1007/978-981-10-3153-3_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-3152-6
Online ISBN: 978-981-10-3153-3
eBook Packages: EngineeringEngineering (R0)