Skip to main content
SpringerLink
Log in

Constructing Uyghur Named Entity Recognition System Using Neural Machine Translation Tag Projection

  • Conference paper
  • First Online:
Chinese Computational Linguistics (CCL 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12522))

Included in the following conference series:

  • 811 Accesses

Abstract

Although named entity recognition achieved great success by introducing the neural networks, it is challenging to apply these models to low resource languages including Uyghur while it depends on a large amount of annotated training data. Constructing a well-annotated named entity corpus manually is very time-consuming and labor-intensive. Most existing methods based on the parallel corpus combined with the word alignment tools. However, word alignment methods introduce alignment errors inevitably. In this paper, we address this problem by a named entity tag transfer method based on the common neural machine translation. The proposed method marks the entity boundaries in Chinese sentence and translates the sentences to Uyghur by neural machine translation system, hope that neural machine translation will align the source and target entity by the self-attention mechanism. The experimental results show that the Uyghur named entity recognition system trained by the constructed corpus achieve good performance on the test set, with 73.80% F1 score (3.79% improvement by baseline).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/HIT-SCIR/pyltp.

  2. 2.

    https://github.com/PaddlePaddle/PaddleHub.

  3. 3.

    https://github.com/thunlp/THULAC-Python.

  4. 4.

    http://ee.dlut.edu.cn/CWMT2017/index_en.html.

  5. 5.

    http://sighan.cs.uchicago.edu/bakeoff2006/.

  6. 6.

    http://uy.ts.cn/.

References

  1. Abiderexiti, K., Maimaiti, M., Yibulayin, T., Wumaier, A.: Annotation schemes for constructing uyghur named entity relation corpus. In: 2016 International Conference on Asian Language Processing (IALP), pp. 103–107. IEEE (2016)

    Google Scholar 

  2. Bai, H., Zhou, Y., Zhang, J., Zhao, L., Hwang, M.Y., Zong, C.: Source-critical reinforcement learning for transferring spoken language understanding to a new language. arXiv preprint arXiv:1808.06167 (2018)

  3. Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Celikyilmaz, A., Choi, Y.: Comet: commonsense transformers for automatic knowledge graph construction. arXiv preprint arXiv:1906.05317 (2019)

  4. Cakır, E., Virtanen, T.: Convolutional recurrent neural networks for rare sound event detection. Deep Neural Networks for Sound Event Detection, vol. 12 (2019)

    Google Scholar 

  5. Casacuberta, F., Vidal, E.: Giza++: training of statistical translation models (2007). Retrieved 29 October 2019

    Google Scholar 

  6. Chen, Y., Zong, C., Su, K.Y.: On jointly recognizing and aligning bilingual named entities. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. pp. 631–639. Association for Computational Linguistics (2010)

    Google Scholar 

  7. Chiu, J.P., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016)

    Article  Google Scholar 

  8. Christopoulou, F., Miwa, M., Ananiadou, S.: A walk-based model on entity graphs for relation extraction. arXiv preprint arXiv:1902.07023 (2019)

  9. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(Aug), 2493–2537 (2011)

    Google Scholar 

  10. Dyer, C., Chahuneau, V., Smith, N.A.: A simple, fast, and effective reparameterization of IBM model 2. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 644–648 (2013)

    Google Scholar 

  11. Ehrmann, M., Turchi, M., Steinberger, R.: Building a multilingual named entity-annotated corpus using annotation projection. In: Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, pp. 118–124 (2011)

    Google Scholar 

  12. Fang, M., Cohn, T.: Learning when to trust distant supervision: An application to low-resource pos tagging using cross-lingual projection. arXiv preprint arXiv:1607.01133 (2016)

  13. Fang, M., Cohn, T.: Model transfer for tagging low-resource languages using a bilingual dictionary. arXiv preprint arXiv:1705.00424 (2017)

  14. Hobbs, J.R., et al.: FASTUS: a cascaded finite-state transducer for extracting information from natural-language text. In: Finite-State Language Processing, pp. 383–406 (1997)

    Google Scholar 

  15. Huang, L., Cho, K., Zhang, B., Ji, H., Knight, K.: Multi-lingual common semantic space construction via cluster-consistent word embedding. arXiv preprint arXiv:1804.07875 (2018)

  16. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)

  17. Kim, S., Toutanova, K., Yu, H.: Multilingual named entity recognition using parallel data and metadata from Wikipedia. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers, vol. 1, pp. 694–702. Association for Computational Linguistics (2012)

    Google Scholar 

  18. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360 (2016)

  19. Li, Z., Wang, X., Ai, A.T., Chng, E.S., Li, H.: Named-entity tagging and domain adaptation for better customized translation (2018)

    Google Scholar 

  20. Lin, Y., Yang, S., Stoyanov, V., Ji, H.: A multi-lingual multi-task architecture for low-resource sequence labeling. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), pp. 799–809 (2018)

    Google Scholar 

  21. Liu, L., et al.: Empower sequence labeling with task-aware neural language model. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  22. Luo, G., Huang, X., Lin, C.Y., Nie, Z.: Joint entity recognition and disambiguation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 879–888 (2015)

    Google Scholar 

  23. Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. arXiv preprint arXiv:1603.01354 (2016)

  24. Maimaiti, M., Wumaier, A., Abiderexiti, K.: Construction of Uyghur named entity corpus. Belt & Road: Language Resources and Evaluation, p. 2 (2018)

    Google Scholar 

  25. Marrero, M., Urbano, J., Sánchez-Cuadrado, S., Morato, J., Gómez-Berbís, J.M.: Named entity recognition: fallacies, challenges and opportunities. Comput. Standards Interfaces 35(5), 482–489 (2013)

    Article  Google Scholar 

  26. Mayhew, S., Tsai, C.T., Roth, D.: Cheap translation for cross-lingual named entity recognition. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2536–2545 (2017)

    Google Scholar 

  27. Ni, J., Dinu, G., Florian, R.: Weakly supervised cross-lingual named entity recognition via effective annotation and representation projection. arXiv preprint arXiv:1707.02483 (2017)

  28. Nothman, J., Ringland, N., Radford, W., Murphy, T., Curran, J.R.: Learning multilingual named entity recognition from Wikipedia. Artif. Intell. 194, 151–175 (2013)

    Article  MathSciNet  Google Scholar 

  29. Östling, R., Tiedemann, J.: Efficient word alignment with Markov chain monte Carlo. Prague Bull. Math. Linguist. 106(1), 125–146 (2016)

    Article  Google Scholar 

  30. Ott, M., et al.: fairseq: a fast, extensible toolkit for sequence modeling. arXiv preprint arXiv:1904.01038 (2019)

  31. Pan, X., Zhang, B., May, J., Nothman, J., Knight, K., Ji, H.: Cross-lingual name tagging and linking for 282 languages. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), pp. 1946–1958 (2017)

    Google Scholar 

  32. Passos, A., Kumar, V., McCallum, A.: Lexicon infused phrase embeddings for named entity resolution. arXiv preprint arXiv:1404.5367 (2014)

  33. Peters, M.E., Ammar, W., Bhagavatula, C., Power, R.: Semi-supervised sequence tagging with bidirectional language models. arXiv preprint arXiv:1705.00108 (2017)

  34. Peters, M.E., et al.: Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)

  35. Ratinov, L., Roth, D.: Design challenges and misconceptions in named entity recognition. In: Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), pp. 147–155 (2009)

    Google Scholar 

  36. Tsai, C.T., Mayhew, S., Roth, D.: Cross-lingual named entity recognition via wikification. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pp. 219–228 (2016)

    Google Scholar 

  37. Ugawa, A., Tamura, A., Ninomiya, T., Takamura, H., Okumura, M.: Neural machine translation incorporating named entity. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 3240–3250 (2018)

    Google Scholar 

  38. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

    Google Scholar 

  39. Wang, D., Peng, N., Duh, K.: A multi-task learning approach to adapting bilingual word embeddings for cross-lingual named entity recognition. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing (vol. 2: Short Papers), pp. 383–388 (2017)

    Google Scholar 

  40. Wang, M., Che, W., Manning, C.D.: Joint word alignment and bilingual named entity recognition using dual decomposition. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), pp. 1073–1082 (2013)

    Google Scholar 

  41. Yang, Z., Salakhutdinov, R., Cohen, W.: Multi-task cross-lingual sequence tagging from scratch. arXiv preprint arXiv:1603.06270 (2016)

  42. Yarowsky, D., Ngai, G., Wicentowski, R.: Inducing multilingual text analysis tools via robust projection across aligned corpora. In: Proceedings of the First International Conference on Human Language Technology Research, pp. 1–8. Association for Computational Linguistics (2001)

    Google Scholar 

  43. Zhou, J.T., Zhang, H., Jin, D., Peng, X., Xiao, Y., Cao, Z.: Roseq: robust sequence labeling. IEEE Trans. Neural Netw. Learn. Syst. (2019)

    Google Scholar 

  44. Žukov-Gregorič, A., Bachrach, Y., Coope, S.: Named entity recognition with parallel recurrent neural networks. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (vol. 2: Short Papers), pp. 69–74 (2018)

    Google Scholar 

Download references

Acknowledgements

This work is supported in part by A Class Funded Project of the Western Light Talent Training Program of the Chinese Academy of Sciences (2017-XBQNXZ-A-005), NSFC (U1703133), The West Light Foundation of The Chinese Academy of Sciences (Grant No. 2019-XBQNXZ-B-008), The National Key R&D Plan (2017YFC0822505-04).

Author information

Authors and Affiliations

  1. Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, Urumqi, China

    Azmat Anwar, Xiao Li, Yating Yang, Rui Dong & Turghun Osman

  2. University of Chinese Academy of Sciences, Beijing, China

    Azmat Anwar, Xiao Li, Yating Yang, Rui Dong & Turghun Osman

  3. Xinjiang Laboratory of Minority Speech and Language Information Processing, Urumqi, China

    Azmat Anwar, Xiao Li, Yating Yang, Rui Dong & Turghun Osman

Authors
  1. Azmat Anwar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yating Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rui Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Turghun Osman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yating Yang .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Maosong Sun

  2. Peking University, Beijing, China

    Sujian Li

  3. Westlake University, Hangzhou, China

    Yue Zhang

  4. Tsinghua University, Beijing, China

    Yang Liu

  5. Chinese Academy of Sciences, Beijing, China

    Shizhu He

  6. Beijing Language and Culture University, Beijing, China

    Gaoqi Rao

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Anwar, A., Li, X., Yang, Y., Dong, R., Osman, T. (2020). Constructing Uyghur Named Entity Recognition System Using Neural Machine Translation Tag Projection. In: Sun, M., Li, S., Zhang, Y., Liu, Y., He, S., Rao, G. (eds) Chinese Computational Linguistics. CCL 2020. Lecture Notes in Computer Science(), vol 12522. Springer, Cham. https://doi.org/10.1007/978-3-030-63031-7_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63031-7_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63030-0

  • Online ISBN: 978-3-030-63031-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation