Skip to main content
Springer Nature Link
Log in

LDA in Character-LSTM-CRF Named Entity Recognition

  • Conference paper
  • First Online:
Text, Speech, and Dialogue (TSD 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11107))

Included in the following conference series:

Abstract

In this paper, we present a NER system based upon deep learning models with character sequence encoding and word sequence encoding in LSTM layers. The results are boosted with LDA topic models and linear-chain CRF sequence tagging. We reach the new state-of-the-art performance in NER of 81.77 F-measure for Czech and 85.91 F-measure Spanish.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Long Short-Term Memory.

  2. 2.

    Convolutional Neural Networks.

  3. 3.

    Gated Recurrent Units.

  4. 4.

    Conditional Random Fields.

  5. 5.

    Latent Dirichlet allocation – see Sect. 2.1.

  6. 6.

    http://trec.nist.gov/data/reuters/reuters.html.

  7. 7.

    Downloaded from https://nlp.stanford.edu/projects/glove/.

  8. 8.

    Downloaded from https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md.

References

  1. Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/, Software available from tensorflow.org

  2. Agerri, R., Rigau, G.: Robust multilingual named entity recognition with shallow semi-supervised features. Artif. Intell. 238, 63–82 (2016)

    Article  MathSciNet  Google Scholar 

  3. Blei, D.M., Ng, A.Y., Jordan, M.I., Lafferty, J.: Latent dirichlet allocation. J. Mach. Learn. Res. 3 (2003)

    Google Scholar 

  4. Chiu, J., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016). http://aclweb.org/anthology/Q16-1026

    Article  Google Scholar 

  5. Konkol, M., Brychcn, T., Konopk, M.: Latent semantics in named entity recognition. Expert Syst. Appl. 42(7), 3470–3479 (2015). https://doi.org/10.1016/j.eswa.2014.12.015, http://www.sciencedirect.com/science/article/pii/S0957417414007933

    Article  Google Scholar 

  6. Konkol, M., Konopík, M.: CRF-based Czech named entity recognizer and consolidation of Czech NER research. In: Habernal, I., Matoušek, V. (eds.) TSD 2013. LNCS (LNAI), vol. 8082, pp. 153–160. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40585-3_20

    Chapter  Google Scholar 

  7. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 260–270. Association for Computational Linguistics (2016). https://doi.org/10.18653/v1/N16-1030, http://www.aclweb.org/anthology/N16-1030

  8. Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany, 7–12 August 2016, vol. 1: Long Papers. The Association for Computer Linguistics (2016). http://aclweb.org/anthology/P/P16/P16-1101.pdf

  9. McCallum, A.K.: Mallet: a machine learning for language toolkit (2002). http://mallet.cs.umass.edu

  10. Nallapati, R., Surdeanu, M., Manning, C.: Blind domain transfer for named entity recognition using generative latent topic models. In: Proceedings of the NIPS 2010 Workshop on Transfer Learning Via Rich Generative Models, pp. 281–289 (2010)

    Google Scholar 

  11. dos Santos, C.N., Guimarães, V.: Boosting named entity recognition with neural character embeddings. In: Duan, X., Banchs, R.E., Zhang, M., Li, H., Kumaran, A. (eds.) Proceedings of the Fifth Named Entity Workshop, NEWS@ACL 2015, Beijing, China, 31 July 2015, pp. 25–33. Association for Computational Linguistics (2015). https://doi.org/10.18653/v1/W15-3904

  12. Straková, J., Straka, M., Hajič, J.: Neural networks for featureless named entity recognition in Czech. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds.) TSD 2016. LNCS (LNAI), vol. 9924, pp. 173–181. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-45510-5_20

    Chapter  Google Scholar 

  13. Tjong Kim Sang, E.F.: Introduction to the CoNLL-2002 shared task: language-independent named entity recognition. In: Proceedings of CoNLL 2002, Taipei, Taiwan, pp. 155–158 (2002)

    Google Scholar 

  14. Ševčíková, M., Žabokrtský, Z., Krůza, O.: Named entities in Czech: annotating data and developing NE tagger. In: Matoušek, V., Mautner, P. (eds.) TSD 2007. LNCS (LNAI), vol. 4629, pp. 188–195. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74628-7_26. http://dl.acm.org/citation.cfm?id=1776334.1776362

    Chapter  Google Scholar 

  15. Yang, Z., Salakhutdinov, R., Cohen, W.W.: Multi-task cross-lingual sequence tagging from scratch. CoRR abs/1603.06270 (2016). http://dblp.uni-trier.de/db/journals/corr/corr1603.html#YangSC16

Download references

Acknowledgements

This work was supported by Ministry of Education, Youth and Sports of the Czech Republic, institutional research support (1311) and by the UWB grant no. SGS-2013-029 Advanced computing and information systems. Access to the MetaCentrum computing facilities provided under the program “Projects of Large Infrastructure for Research, Development, and Innovations” LM2010005, funded by the Ministry of Education, Youth, and Sports of the Czech Republic, is highly appreciated.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Faculty of Applied Sciences, University of West Bohemia, Univerzitní 8, 306 14, Plzeň, Czech Republic

    Miloslav Konopík & Ondřej Pražák

Authors
  1. Miloslav Konopík
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ondřej Pražák
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miloslav Konopík .

Editor information

Editors and Affiliations

  1. Faculty of Informatics, Masaryk University, Brno, Czech Republic

    Petr Sojka

  2. Faculty of Informatics, Masaryk University, Brno, Czech Republic

    Aleš Horák

  3. Faculty of Informatics, Masaryk University, Brno, Czech Republic

    Ivan Kopeček

  4. Faculty of Informatics, Masaryk University, Brno, Czech Republic

    Karel Pala

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Konopík, M., Pražák, O. (2018). LDA in Character-LSTM-CRF Named Entity Recognition. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds) Text, Speech, and Dialogue. TSD 2018. Lecture Notes in Computer Science(), vol 11107. Springer, Cham. https://doi.org/10.1007/978-3-030-00794-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00794-2_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00793-5

  • Online ISBN: 978-3-030-00794-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics