Abstract
In this paper, we present a Long Short-Term Memory (LSTM) based model for the task of Chinese Weibo word segmentation. The model adopts a LSTM layer to capture long-range dependencies in sentence and learn the underlying patterns. In order to infer the optimal tag path, we introduce a transition score matrix for jumping between tags of successive characters. Integrated with some unsupervised features, the performance of the model is further improved. Finally, our model achieves a weighted F1-score of 0.8044 on close track, 0.8298 on the semi-open track.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Conference on Natural Language Processing and Chinese Computing. http://tcci.ccf.org.cn/conference/2016.
- 2.
A weighted evaluation metric provided by organizers that gives more reasonable and distinguishable scores and correlates well with human judgment. More details in Qian et al. [8].
- 3.
- 4.
- 5.
- 6.
Chinese idiom dictionary is used. https://github.com/sunflowerlyb/idiom.
- 7.
References
Qiu, X., Qian, P., Shi, Z., Wu, S.: Overview of the NLPCC 2016 shared task: Chinese word segmentation for micro-blog texts (2016)
Yu, Z., Dai, X.-Y., Shen, S., Huang, S., Chen, J.: Word segmentation of micro blogs with bagging. In: Li, J., Ji, H., Zhao, D., Feng, Y. (eds.) NLPCC 2015. LNCS (LNAI), vol. 9362, pp. 573–580. Springer, Heidelberg (2015). doi:10.1007/978-3-319-25207-0_54
Min, K., Ma, C., Zhao, T., Li, H.: BosonNLP: an ensemble approach for word segmentation and POS tagging. In: Li, J., Ji, H., Zhao, D., Feng, Y. (eds.) NLPCC 2015. LNCS (LNAI), vol. 9362, pp. 520–526. Springer, Heidelberg (2015). doi:10.1007/978-3-319-25207-0_48
Pei, W., Ge, T., Chang, B.: Max-margin tensor neural network for Chinese word segmentation. In: ACL, vol. 1, pp. 293–303 (2014)
Chen, X., Qiu, X., Zhu, C., Huang, X.: Gated recursive neural network for Chinese word segmentation. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (2015a)
Chen, X., Qiu, X., Zhu, C., Liu, P., Huang, X.: Long short-term memory neural networks for Chinese word segmentation (2015b)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Qian, P., Qiu, X., Huang, X.: A new psychometric-inspired evaluation metric for Chinese word segmentation. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL) (2016)
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)
Goller, C., Kuchler, A.: Learning task-dependent distributed representations by back propagation through structure. In: IEEE International Conference on Neural Networks, vol. 1, pp. 347–352. IEEE, June 1996
Ghahramani, Z.: An introduction to hidden Markov models and Bayesian networks. Int. J. Pattern Recogn. Artif. Intell. 15(01), 9–42 (2001)
Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bengio, Y.: Theano: new features and speed improvements (2012). arXiv preprint arXiv:1211.5590
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Bengio, Y.: Theano: a CPU and GPU math expression compiler. In: Proceedings of Python for Scientific Computing Conference (SciPy), vol. 4, p. 3, June 2010
Feng, H., Chen, K., Deng, X., Zheng, W.: Accessor variety criteria for Chinese word extraction. Comput. Linguist. 30(1), 75–93 (2004)
Wu, G., He, D., Zhong, K., Zhou, X., Yuan, C.: Leveraging rich linguistic features for cross-domain Chinese segmentation. In: CLP 2014, p. 101 (2014)
Gao, Q., Vogel, S.: A multi-layer Chinese word segmentation system optimized for out-of-domain tasks. In: Proceedings of CIPS-SIGHAN Joint Conference on Chinese Language Processing (CLP 2010), pp. 210–215 (2010)
Sun, W., Xu, J.: Enhancing Chinese word segmentation using unlabeled data. In: Proceedings of Conference on Empirical Methods in Natural Language Processing, pp. 970–979. Association for Computational Linguistics, July 2011
Qiu, X., Qian, P., Yin, L., Wu, S., Huang, X.: Overview of the NLPCC 2015 shared task: Chinese word segmentation and POS tagging for micro-blog texts. In: Li, J., Ji, H., Zhao, D., Feng, Y. (eds.) NLPCC 2015. LNCS (LNAI), vol. 9362, pp. 541–549. Springer, Heidelberg (2015). doi:10.1007/978-3-319-25207-0_50
Acknowledgments
This work was partially supported by Natural Science Foundation of China (Nos. 61273365, 61202248), discipline building plan in 111 base (No. B08004) and Engineering Research Center of Information Networks of MOE, and the Co-construction Program with the Beijing Municipal Commission of Education. Many thanks to Caixia Yuan and Guohua Wu for their insightful comments. We are also very thankful to Xipeng Qiu for his excellent organization.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Zhou, Q., Ma, L., Zheng, Z., Wang, Y., Wang, X. (2016). Recurrent Neural Word Segmentation with Tag Inference. In: Lin, CY., Xue, N., Zhao, D., Huang, X., Feng, Y. (eds) Natural Language Understanding and Intelligent Applications. ICCPOL NLPCC 2016 2016. Lecture Notes in Computer Science(), vol 10102. Springer, Cham. https://doi.org/10.1007/978-3-319-50496-4_66
Download citation
DOI: https://doi.org/10.1007/978-3-319-50496-4_66
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50495-7
Online ISBN: 978-3-319-50496-4
eBook Packages: Computer ScienceComputer Science (R0)