Abstract
Entity Linking (EL) refers to the task of linking entity mentions in the text to the correct entities in the Knowledge Base (KB) in which entity embeddings play a vital and challenging role because of the subtle differences between entities. However, existing pre-trained entity embeddings only learn the underlying semantic information in texts, yet the fine-grained entity type information is ignored, which causes the type of the linked entity is incompatible with the mention context. In order to solve this problem, we propose to encode fine-grained type information into entity embeddings. We firstly pre-train word vectors to inject type information by embedding words and fine-grained entity types into the same vector space. Then we retrain entity embeddings with word vectors containing fine-grained type information. By applying our entity embeddings to two existing EL models, our method respectively achieves 0.82\(\%\) and 0.42\(\%\) improvement on average F1 score of the test sets. Meanwhile, our method is model-irrelevant, which means it can help other EL models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cao, Y., Hou, L., Li, J., Liu, Z.: Neural collective entity linking. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 675–686. Association for Computational Linguistics, Santa Fe (2018). https://www.aclweb.org/anthology/C18-1057
Chen, S., Wang, J., Jiang, F., Lin, C.: Improving entity linking by modeling latent entity type information. CoRR abs/2001.01447 (2020). http://arxiv.org/abs/2001.01447
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805 (2018). http://arxiv.org/abs/1810.04805
Eshel, Y., Cohen, N., Radinsky, K., Markovitch, S., Yamada, I., Levy, O.: Named entity disambiguation for noisy text. In: Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pp. 58–68. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/K17-1008, https://www.aclweb.org/anthology/K17-1008
Ganea, O.E., Hofmann, T.: Deep joint entity disambiguation with local neural attention. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2619–2629. Association for Computational Linguistics, Copenhagen (2017). https://doi.org/10.18653/v1/D17-1277, https://www.aclweb.org/anthology/D17-1277
Gupta, N., Singh, S., Roth, D.: Entity linking via joint encoding of types, descriptions, and context. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2681–2690. Association for Computational Linguistics, Copenhagen (2017). https://doi.org/10.18653/v1/D17-1284, https://www.aclweb.org/anthology/D17-1284
Hoffart, J., et al.: Robust disambiguation of named entities in text. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 782–792. Association for Computational Linguistics, Edinburgh (2011). https://www.aclweb.org/anthology/D11-1072
Hou, F., Wang, R., He, J., Zhou, Y.: Improving entity linking through semantic reinforced entity embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6843–6848. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.612, https://www.aclweb.org/anthology/2020.acl-main.612
Lazic, N., Subramanya, A., Ringgaard, M., Pereira, F.: Plato: a selective context model for entity resolution. Trans. Assoc. Comput. Linguist. 3(1), 503–515 (2015)
Le, P., Titov, I.: Improving entity linking by modeling latent relations between mentions. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1595–1604. Association for Computational Linguistics, Melbourne (2018). https://doi.org/10.18653/v1/P18-1148, https://www.aclweb.org/anthology/P18-1148
Le, P., Titov, I.: Boosting entity linking performance by leveraging unlabeled documents. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1935–1945. Association for Computational Linguistics, Florence (2019). https://doi.org/10.18653/v1/P19-1187, https://www.aclweb.org/anthology/P19-1187
Ling, X., Weld, D.S.: Fine-grained entity recognition. In: Proceedings of the 26th AAAI Conference on Artificial Intelligence (2012)
Logeswaran, L., Chang, M.W., Lee, K., Toutanova, K., Devlin, J., Lee, H.: Zero-shot entity linking by reading entity descriptions. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3449–3460. Association for Computational Linguistics, Florence (2019). https://doi.org/10.18653/v1/P19-1335, https://www.aclweb.org/anthology/P19-1335
Maaten, L.J.P.V.D.: Accelerating t-SNE using tree-based algorithms. J. Mach. Learn. Res. 15, 3221–3245 (2014)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. Computer Science (2013)
Rijhwani, S., Xie, J., Neubig, G., Carbonell, J.: Zero-shot neural transfer for cross-lingual entity linking. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 6924–6931 (2019)
Sevgili, Ö., Panchenko, A., Biemann, C.: Improving neural entity disambiguation with graph embeddings. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pp. 315–322. Association for Computational Linguistics, Florence (2019). https://doi.org/10.18653/v1/P19-2044, https://www.aclweb.org/anthology/P19-2044
Yamada, I., Shindo, H., Takeda, H., Takefuji, Y.: Joint learning of the embedding of words and entities for named entity disambiguation. In: Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pp. 250–259. Association for Computational Linguistics, Berlin (2016). https://doi.org/10.18653/v1/K16-1025, https://www.aclweb.org/anthology/K16-1025
Yang, X., et al.: Learning dynamic context augmentation for global entity linking. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 271–281. Association for Computational Linguistics, Hong Kong (2019). https://doi.org/10.18653/v1/D19-1026, https://www.aclweb.org/anthology/D19-1026
Zhou, B., Khashabi, D., Tsai, C.T., Roth, D.: Zero-shot open entity typing as type-compatible grounding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2065–2076. Association for Computational Linguistics, Brussels (2018). https://doi.org/10.18653/v1/D18-1231, https://www.aclweb.org/anthology/D18-1231
Zhou, S., Rijhwani, S., Wieting, J., Carbonell, J., Neubig, G.: Improving candidate generation for low-resource cross-lingual entity linking. Trans. Assoc. Comput. Linguist. 8, 109–124 (2020)
Acknowledgements
We are grateful for helpful comments and suggestions from the anonymous reviewers. This work is supported by the National Nature Science Foundation of China (Contract 61876198, 61976015, 61976016).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, T., Yang, E., Zhang, Y., Xu, J., Chen, Y. (2021). Improving Entity Linking by Encoding Type Information into Entity Embeddings. In: Li, S., et al. Chinese Computational Linguistics. CCL 2021. Lecture Notes in Computer Science(), vol 12869. Springer, Cham. https://doi.org/10.1007/978-3-030-84186-7_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-84186-7_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84185-0
Online ISBN: 978-3-030-84186-7
eBook Packages: Computer ScienceComputer Science (R0)