Abstract
Vietnamese is very different from English and little research has been done on Vietnamese document classification, or indeed, on any kind of Vietnamese language processing, and only a few small corpora are available for research. We created a large Vietnamese text corpus with about 18000 documents, and manually classified them based on different criteria such as topics and styles, giving several classification tasks of different difficulty levels. This paper introduces a new syllable-based document representation at the morphological level of the language for efficient classification. We tested the representation on our corpus with different classification tasks using six classification algorithms and two feature selection techniques. Our experiments show that the new representation is effective for Vietnamese categorization, and suggest that best performance can be achieved using syllable-pair document representation, an SVM with a polynomial kernel as the learning algorithm, and using Information gain and an external dictionary for feature selection.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sebastiani, F.: Machine learning in automated text categorization. ACM Computing Surveys 34(1), 1–47 (2002)
Dinh, D., Hoang, K.: Vietnamese Word Segmentation. In: Proceedings of Sixth Natural Language Processing Pacific Rim Symposium (NLPRS 2001), Tokyo (2001)
Dinh, D.: POS-Tagger for English – Vietnamese Bilingual Corpus. In: Workshop: Building and Using Parallel Texts: Data Driven Machine Translation and Beyond, Edmonton, CA (2003)
Nguyen, T.M.H., Vu, X.L.: Une etude de cas pour l’etiquetage morpho-syntaxique de texts vietnamiens. In: The TALN Conference, Batz-sur-mer, France (2003)
Nguyen, T.B.: Lexical descriptions for Vietnamese language processing. In: The 1st International Joint Conference on Natural Language Processing, Workshop on Asian Language Resource, Sanya, Hainan Island, China (2004)
Vu, D.: The Vietnamese and contemporary languages – an overview of syntax (in Vietnamese), Viet Stuttgart, Germany (2004)
Nguyen, T.C.: Vietnamese Grammar. NXB Đại học Qu’̂̂oc Gia, Hanoi (1998)
Cao, X. H.: The Vietnamese – some problems about phonetics, grammar, and semantics, (in Vietnamese). NXB Giáo dụç, Hanoi (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nguyen, GS., Gao, X., Andreae, P. (2009). Vietnamese Document Representation and Classification. In: Nicholson, A., Li, X. (eds) AI 2009: Advances in Artificial Intelligence. AI 2009. Lecture Notes in Computer Science(), vol 5866. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10439-8_58
Download citation
DOI: https://doi.org/10.1007/978-3-642-10439-8_58
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-10438-1
Online ISBN: 978-3-642-10439-8
eBook Packages: Computer ScienceComputer Science (R0)