Abstract
Discrete and Neural models are two mainstream methods for Chinese POS tagging nowadays. Both have achieved state-of-the-art performances. In this paper, we compare the two kinds of models empirically, and further investigate the combination methods of them. In particular, as the pre-trained word embeddings are exploited under the neural setting, one can regard neural models as semi-supervised setting. To make a fairer comparison of the discrete and the neural models, we incorporate word clusters for both models as well as their combination, since it has been generally accepted that word clusters can encode similar information as pre-trained word embeddings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
https://sourceforge.net/projects/zpar, version 0.7.
- 2.
- 3.
We use the tool available at https://github.com/percyliang/brown-cluster to produce word clusters, using the same corpus as the training of word embeddings.
References
Brants, T.: TnT: a statistical part-of-speech tagger. In: Proceedings of the Sixth Conference on Applied Natural Language Processing, pp. 224–231 (2000)
Collins, M.: Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms. In: EMNLP (2002)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. JMLR 12, 2493–2537 (2011)
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. JMLR 12, 2121–2159 (2011)
Giménez, J., Marquez, L.: SVMTool: a general POS tagger generator based on support vector machines. In: Proceedings of the 4th LREC (2004)
Hatori, J., Matsuzaki, T., Miyao, Y., Tsujii, J.: Incremental joint POS tagging and dependency parsing in Chinese. In: IJCNLP, pp. 1216–1224 (2011)
Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: ICML, pp. 282–289 (2001)
Li, Z., Chao, J., Zhang, M., Chen, W.: Coupled sequence labeling on heterogeneous annotations: POS tagging as a case study. In: ACL, pp. 1783–1792 (2015)
Li, Z., Che, W., Liu, T.: Improving Chinese POS tagging with dependency parsing. In: IJCNLP, pp. 1447–1451 (2011)
Li, Z., Liu, T., Che, W.: Exploiting multiple treebanks for parsing with quasi-synchronous grammars. In: Proceedings of the 50th ACL, pp. 675–684 (2012)
Li, Z., Zhang, M., Che, W., Liu, T.: A separately passive-aggressive training algorithm for joint POS tagging and dependency parsing. In: COLING 2012, pp. 1681–1698 (2012)
Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNS-CRF. In: ACL (2016)
Manning, C.D.: Part-of-speech tagging from 97% to 100%: is it time for some linguistics? In: Gelbukh, A.F. (ed.) CICLing 2011. LNCS, vol. 6608, pp. 171–189. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19400-9_14
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
dos Santos, C.N., Zadrozny, B.: Learning character-level representations for part-of-speech tagging. In: ICML, pp. 1818–1826 (2014)
Sun, W.: Word-based and character-based word segmentation models: comparison and combination. In: COLING 2010: Posters, pp. 1211–1219 (2010)
Sun, W., Uszkoreit, H.: Capturing paradigmatic and syntagmatic lexical relations: towards accurate Chinese part-of-speech tagging. In: ACL 2012, pp. 242–252 (2012)
Sun, W., Wan, X.: Data-driven, PCFG-based and pseudo-PCFG-based models for Chinese dependency parsing. TACL 1(1), 301–314 (2013)
Toutanova, K., Klein, D., Manning, C., Singer, Y.: Feature-rich part-of-speech tagging with a cyclic dependency network. In: HLT-NAACL 2003 (2003)
Wang, Y., Kazama, J., Tsuruoka, Y., Chen, W., Zhang, Y., Torisawa, K.: Improving Chinese word segmentation and POS tagging with semi-supervised methods using large auto-analyzed data. In: IJCNLP, pp. 309–317 (2011)
Wolpert, D.H.: Stacked generalization. Neural Netw. 5, 241–259 (1992)
Zhang, M., Che, W., Liu, T., Li, Z.: Stacking heterogeneous joint models of Chinese POS tagging and dependency parsing. In: COLING 2012, pp. 3071–3088 (2012)
Zhang, M., Yang, J., Teng, Z., Zhang, Y.: LibN3L: a lightweight package for neural NLP. In: LREC (2016)
Zhang, M., Zhang, Y.: Combining discrete and continuous features for deterministic transition-based dependency parsing. In: EMNLP, pp. 1316–1321 (2015)
Zhang, M., Zhang, Y., Fu, G.: Transition-based neural word segmentation. In: Proceedings of the 54nd ACL (2016)
Zhang, M., Zhang, Y., Vo, D.T.: Neural networks for open domain targeted sentiment. In: Proceedings of the EMNLP, pp. 612–621 (2015)
Acknowledgments
This work is supported by National Natural Science Foundation of China (NSFC) under grant 61602160 and 61672211, Natural Science Foundation of Heilongjiang Province (China) under grant No. F2016036.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Zhang, M., Yu, N., Fu, G. (2016). Discrete and Neural Models for Chinese POS Tagging: Comparison and Combination. In: Lin, CY., Xue, N., Zhao, D., Huang, X., Feng, Y. (eds) Natural Language Understanding and Intelligent Applications. ICCPOL NLPCC 2016 2016. Lecture Notes in Computer Science(), vol 10102. Springer, Cham. https://doi.org/10.1007/978-3-319-50496-4_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-50496-4_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50495-7
Online ISBN: 978-3-319-50496-4
eBook Packages: Computer ScienceComputer Science (R0)