Abstract
A number of language-independent text pre-processing techniques, to support multi-class single-label text classification, are described and compared. A simple but effective statistical keyword identification approach is proposed, coupled with a number of phrase identification mechanisms. Experimental results are presented.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Coenen, F., Leng, P. & Zhang, L. Threshold tuning for improved classification association rule mining. In: Ho, T.B., Cheung, D. & Liu, H. (ed) Proceedings of the 9th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2005), Hanoi, Vietnam, 2005 (LNAI 3518, Springer, pp. 216–225)
Collier, N., Nobata, C. & Tsujii, J. Extracting the names of genes and gene products with a hidden markov model. In: Proceedings of the 18th International Conference on Computational Linguistics (COLING 2000), Saarbrücken, Germany, 2000 (pp. 201–207)
French, J.C., Powell, A.L., Callan, J., Viles, CL., Emmitt, T., Prey, K.J. & Mou, Y. Comparing the performance of database selection algorithms. Technical report CS-99-03, Department of Computer Science, University of Virginia, January 1999
Kazama, J., Makino, T., Ohta, Y. & Tsujii, J. Tuning support vector machines for biomedical named entity recognition. In: Proceedings of the ACL (Association for Computational Linguistics) Workshop on Natural Language Processing in the Biomedical Domain (ACL 2002), Philadelphia, PA, USA, 2002 (pp. 1–8)
Lang, K. Newsweeder: learning to filter netnews. In: Proceedings of the 12th International Conference on Machine Learning (ICML 1995), Tahoe City, California, USA, 1995 (pp. 331–339)
Li, X. & Liu, B. Learning to classify texts using positive and unlabeled data. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI-03), Acapulco, Mexico, 2003 (pp. 587–594)
Mladenic, D. & Grobelnik, M. Word sequences as features in text-learning. In: Proceedings of the 7th Electrotechnical and Computer Security Conference (ERK 1998), IEEE Region 8, Slovenia Section IEEE, Ljubligana, Slovenia, 1998 (pp. 145–148)
Nobata, C, Collier, N. & Tsujii, J. Automatic term identification and classification in biological texts. In: Proceedings of the 5th Natural Language Pacific Rim Symposium (NLPRS 1999), Beijing, China, 1999 (pp. 369–375)
Spärck Jones, K. Exhaustivity and specificity. Journal of Documentation 1972; 28:11–21 (reprinted in 2004; 60:493–502)
Yang, Y. & Pedersen, J.O. A comparative study on feature set selection in text categorization. In: Proceedings of the 14th International Conference on Machine Learning (ICML 1997), Nashville, TN, USA, 1997 (pp. 412–420)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag London Limited
About this paper
Cite this paper
Wang, Y.J., Coenen, F., Leng, P., Sanderson, R. (2007). Text Classification using Language-independent Pre-processing. In: Bramer, M., Coenen, F., Tuson, A. (eds) Research and Development in Intelligent Systems XXIII. SGAI 2006. Springer, London. https://doi.org/10.1007/978-1-84628-663-6_34
Download citation
DOI: https://doi.org/10.1007/978-1-84628-663-6_34
Publisher Name: Springer, London
Print ISBN: 978-1-84628-662-9
Online ISBN: 978-1-84628-663-6
eBook Packages: Computer ScienceComputer Science (R0)