Abstract
This paper deals with the task of part-of-speech (POS) tagging for Vietnamese social media text, which poses several challenges compared with tagging for conventional text. We introduce a POS tagging model that takes advantages of deep learning and manually engineered features to overcome the challenges of the task. The main part of the model consists of several bidirectional long short-term memory (BiLSTM) layers that are used to learn intermediate representations of sentences from features extracted at both the character and the word levels. Conditional random field (CRF) is then used on top of those BiLSTM layers, at the inference layer, to predict the most suitable POS tags. We leverage various types of manually engineered features in addition to automatically learned features to capture the characteristics of Vietnamese social media data and therefore improve the performance of the model. Experimental results on a public POS tagging corpus for Vietnamese social media text show that our model outperforms previous work [4] by a large margin, reaching 91.9% accuracy with 27% error rate reduction. The results also reveal the effectiveness of using both automatically learned and manually designed features in a deep learning framework when only a limited amount of training data is available.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Goldberg [11] shows that by stacking several BiLSTM layers we can produce better representations for sentences.
- 2.
- 3.
The division into training and test sets is the same as the work of Bach et al. [4].
- 4.
Software available at https://github.com/jiesutd/NCRFpp.
References
Albogamy, F., Ramsay A.: POS tagging for Arabic tweets. In: Proceedings of RANLP, pp. 1–8 (2015)
Albogamy, F., Ramsay, A.: Fast and robust POS tagger for Arabic tweets using agreement-based bootstrapping. In: Proceedings of LREC, pp. 1500–1506 (2016)
Bach, N.X., Hiraishi, K., Minh, N.L., Shimazu, A.: Dual decomposition for Vietnamese part-of-speech tagging. In: Proceedings of KES, pp. 123–131 (2013)
Bach, N.X., Linh, N.D., Phuong, T.M.: An empirical study on POS tagging for Vietnamese social media text. Comput. Speech Lang. 50, 1–15 (2018)
Brants, T.: TnT: a statistical part-of-speech tagger. In: Proceedings of the Sixth Conference on Applied Natural Language Processing, pp. 224–231 (2000)
Brown, P.F., Desouza, P.V., Mercer, R.L., Pietra, V.D., Lai, J.: Class-based n-gram models of natural language. Comput. Linguist. 18(4), 467–479 (1992)
Derczynski, L., Ritter, A., Clark, S., Bontcheva, K.: Twitter part-of-speech tagging for all: overcoming sparse and noisy data. In: Proceedings of the RANLP, pp. 198–206 (2013)
Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)
Gimenez, J., Marquez, L.: SVM-tool: a general POS tagger generator based on support vector machines. In: Proceedings of LREC, pp. 43–46 (2004)
Gimpel, K. et al.: Part-of-speech tagging for twitter: annotation, features, and experiments. In: Proceedings of ACL, pp. 42–47 (2011)
Goldberg, Y.: Neural Network Methods for Natural Language Processing. Morgan & Claypool, San Rafael (2017)
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arXiv:1512.03385 (2015). https://arxiv.org/abs/1512.03385
Heigold, G., Neumann, G., Genabith, J.V.: An extensive empirical evaluation of character-based morphological tagging for 14 languages. In: Proceedings of EACL, pp. 505–513 (2017)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv:1508.01991. https://arxiv.org/abs/1508.01991 (2015)
Labeau, M., Loser, K., Allauzen, A.: Non-lexical neural architecture for fine-grained POS tagging. In: Proceedings of EMNLP, pp. 232–237 (2015)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of ICML, pp. 282–289 (2001)
Le, H.P., Roussanaly, A., Nguyen, T.M.H., Rossignol, M.: An empirical study of maximum entropy approach for part-of-speech tagging of Vietnamese texts. In: Proceedings of the TALN (2010)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(110), 2278–2324 (1998)
LeCun, Y., Bengio, Y., Hinton, G.: Deep Learning. Nature 521, 436–444 (2015)
Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of ACL, pp. 1064–1074 (2016)
Neunerdt, M., Trevisan, B., Reyer, M., Mathar, R.: Part-of-speech tagging for social media texts. In: Gurevych, I., Biemann, C., Zesch, T. (eds.) GSCL 2013. LNCS (LNAI), vol. 8105, pp. 139–150. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40722-2_15
Nghiem, M., Dinh, D., Nguyen, M.: Improving Vietnamese POS tagging by integrating a rich feature set and support vector machines. In: Proceedings of RIVF, pp. 128–133 (2008)
Nguyen, L.M., Ngo, X.B., Nguyen, V.C., Pham, Q.N.M., Shimazu, A.: A semi-supervised learning method for Vietnamese part-of-speech tagging. In: Proceedings of KSE, pp. 141–146 (2010)
Nguyen, D.Q., Vu, T., Nguyen, D.Q., Dras, M., Johnson, M.: From word segmentation to POS tagging for Vietnamese. In: Proceedings of ALTA, pp. 108–113 (2017)
Nooralahzadeh, F., Brun, C., Roux, C.: Part of speech tagging for French social media data. In: Proceedings of COLING, pp. 1764–1772 (2014)
Owoputi, O., O’Connor, B., Dyer, C., Gimpel, K., Schneider, N., Smith, N.A.: Improved part-of-speech tagging for online conversational text with word clusters. In: Proceedings of NAACL, pp. 380–390 (2013)
Plank, B., Sogaard, A., Goldberg, Y.: Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In: Proceedings of ACL, pp. 412–418 (2016)
Ratnaparkhi, A.: A maximum entropy model for part-of-speech tagging. In: Proceedings of EMNLP, pp. 133–142 (1996)
Rehbein, I.: Fine-grained POS tagging of German tweets. In: Gurevych, I., Biemann, C., Zesch, T. (eds.) GSCL 2013. LNCS (LNAI), vol. 8105, pp. 162–175. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40722-2_17
Shao, Y., Hardmeier, C., Tiedemann, J., Nivre, J.: Character-based joint segmentation and POS tagging for Chinese using bidirectional RNN-CRF. In: Proceedings of IJCNLP, pp. 173–183 (2017)
Stratos, K., Kim, D., Collins, M., Hsu, D.: A spectral algorithm for learning class-based n-gram models of natural language. In: Proceedings of UAI, pp. 762–771 (2014)
Stratos, K., Collins, M.: Simple semi-supervised POS tagging. In: Proceedings of NAACL-HLT, pp. 79–87 (2015)
Toutanova, K., Manning, C.: Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In: Proceedings of EMNLP, pp. 63–70 (2000)
Tran, T.O., Le, A.C., Ha, Q.T., Le, H.Q.: An experimental study on Vietnamese POS tagging. In: Proceedings of IALP, pp. 23–27 (2009)
Wang, P., Qian, Y., Soong, F.K., He, L., Zhao, H.: Part-of-speech tagging with bidirectional long short-term memory recurrent neural network. arXiv:1510.06168 (2015). https://arxiv.org/abs/1510.06168
Yang, J., Zhang, Y.: NCRF++: an open-source neural sequence labeling toolkit. In: Proceedings of ACL-System Demonstrations, pp. 74–79 (2018)
Young, T., Hazarika, D., Poria, S., Cambria, E.: Recent trends in deep learning based natural language processing. arXiv:1708.02709v8 (2018). https://arxiv.org/abs/1708.02709
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Xuan Bach, N., Khuong Duy, T., Minh Phuong, T. (2019). A POS Tagging Model for Vietnamese Social Media Text Using BiLSTM-CRF with Rich Features. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11672. Springer, Cham. https://doi.org/10.1007/978-3-030-29894-4_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-29894-4_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29893-7
Online ISBN: 978-3-030-29894-4
eBook Packages: Computer ScienceComputer Science (R0)