Abstract
Character-based Chinese dependency parsing jointly learns Chinese word segmentation, POS tagging and dependency parsing to avoid the error propagation problem of pipeline models. Recent works on this task only rely on a local status for prediction at each step, which is insufficient for guiding global better decisions. In this paper, we first present a sequence-to-action model for character-based dependency parsing. In order to exploit decision history for prediction, our model tracks the status of parser particularly including decision history in the decoding procedure by employing a sequential LSTM. Additionally, for resolving the problem of high ambiguities in Chinese characters, we add position-based character embeddings to exploit character information with specific contexts accurately. We conduct experiments on Penn Chinese Treebank 5.1 (CTB-5) dataset, and the results show that our proposed model outperforms existing neural network system in dependency parsing, and performs preferable accuracy in Chinese word segmentation and POS tagging.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Hatori, J., Matsuzaki, T., Miyao, Y., Tsujii, J.I.: Incremental joint approach to word segmentation, POS tagging, and dependency parsing in Chinese. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pp. 1045–1053. Association for Computational Linguistics (2012)
Nivre, J. An efficient algorithm for projective dependency parsing. In: Proceedings of the Eighth International Conference on Parsing Technologies, pp. 149–160 (2003)
Zhang, M., Zhang, Y., Che, W., Liu, T.: Character-level Chinese dependency parsing. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, vol. 1, pp. 1326–1336 (2014)
Guo, Z., Zhang, Y., Su, C., Xu, J., Isahara, H.: Character-level dependency model for joint word segmentation, POS tagging, and dependency parsing in Chinese. IEICE Trans. Inf. Syst. 99(1), 257–264 (2016)
Kurita, S., Kawahara, D., Kurohashi, S.: Neural joint model for transition-based Chinese syntactic analysis. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, pp. 1204–1214 (2017)
Li, H., Zhang, Z., Ju, Y., Zhao, H.: Neural character-level dependency parsing for Chinese. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Dyer, C., Ballesteros, M., Ling, W., Matthews, A., Smith, N.A.: Transition-based dependency parsing with stack long short-term memory. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Volume 1: Long Papers, vol. 1, pp. 334–343 (2015)
Bowman, S.R., Gauthier, J., Rastogi, A., Gupta, R., Manning, C.D., Potts, C.: A fast unified model for parsing and sentence understanding. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, vol. 1, pp. 1466–1477 (2016)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, vol. 2, pp. 3104–3112. MIT Press (2014)
Nivre, J.: Incrementality in deterministic dependency parsing. In: Proceedings of the Workshop on Incremental Parsing: Bringing Engineering and Cognition Together, pp. 50–57. Association for Computational Linguistics (2004)
Chen, X., Xu, L., Liu, Z., Sun, M., Luan, H.: Joint learning of character and word embeddings. In: Proceedings of the 24th International Conference on Artificial Intelligence, pp. 1236–1242. AAAI Press (2015)
Dozat, T., Manning, C.D.: Deep biaffine attention for neural dependency parsing. arXiv preprint arXiv:1611.01734 (2016)
Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Volume 1: Long Papers, vol. 1, pp. 1556–1566 (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Jiang, W., Huang, L., Liu, Q., Lv, Y.: A cascaded linear model for joint Chinese word segmentation and part-of-speech tagging. In: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics (2008)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Peters, M., et al.: Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1: Long Papers, pp. 2227–2237 (2018)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Acknowledgments
The authors are supported by the National Nature Science Foundation of China (61876198, 61370130 and 61473294).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, H., Zhang, Y., Chen, M., Xu, J., Chen, Y. (2019). A Sequence-to-Action Architecture for Character-Based Chinese Dependency Parsing with Status History. In: Tang, J., Kan, MY., Zhao, D., Li, S., Zan, H. (eds) Natural Language Processing and Chinese Computing. NLPCC 2019. Lecture Notes in Computer Science(), vol 11839. Springer, Cham. https://doi.org/10.1007/978-3-030-32236-6_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-32236-6_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32235-9
Online ISBN: 978-3-030-32236-6
eBook Packages: Computer ScienceComputer Science (R0)
