Abstract
To achieve a better performance in the downstream task of knowledge graph (KG), a good representation of KG is necessary. Sensing from the topological structure of the graph, most conventional methods tend to ignore the semantic features of nodes, which is significant for describing the entity in KG. In this paper, we propose a novel Knowledge Graph Embedding method based on Semantics and Structure (KGESS), which learned the representation of KG from both topological facts and semantic information. It leverages Chinese BERT to obtain semantic features of the entity first. Then it further enhances these features via a neural module, namely Semantic Feature Extractor. To evaluate the performance of KGESS, we utilize an additional linear module to execute the link prediction task. Experimental results demonstrate that KGESS achieves a superior Hit@k score than conventional methods, indicating the effectiveness of the idea of enhancing structure with semantics in the representation task of KG.
X. Chen and Z. Ma—Co-author
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Process. Syst. 26 (2013)
Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Celikyilmaz, A., Choi, Y.: COMET: commonsense transformers for automatic knowledge graph construction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4762–4779. Association for Computational Linguistics, Florence, Italy, July 2019. https://doi.org/10.18653/v1/P19-1470, https://aclanthology.org/P19-1470
Danqi, C., Richard, S., et al.: Learning new facts from knowledge bases with neural tensor networks and semantic word vectors. Comput. Sci. 1, 392–399 (2013)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota, June 2019. https://doi.org/10.18653/v1/N19-1423, https://aclanthology.org/N19-1423
Gao, Tianyu, Zhang, Yuanming, Li, Mengni, Lu, Jiawei, Cheng, Zhenbo, Xiao, Gang: Representation learning of knowledge graph with semantic vectors. In: Qiu, Han, Zhang, Cheng, Fei, Zongming, Qiu, Meikang, Kung, Sun-Yuan. (eds.) KSEM 2021. LNCS (LNAI), vol. 12816, pp. 16–29. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-82147-0_2
Hu, F., Lakdawala, S., Hao, Q., Qiu, M.: Low-power, intelligent sensor hardware interface for medical data preprocessing. IEEE Trans. Inf Technol. Biomed. 13(4), 656–663 (2009)
Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (vol. 1: Long Papers), pp. 687–696 (2015)
Li, Y., Song, Y., Jia, L., Gao, S., Li, Q., Qiu, M.: Intelligent fault diagnosis by fusing domain adversarial training and maximum mean discrepancy via ensemble learning. IEEE Trans. Industr. Inf. 17(4), 2833–2841 (2020)
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Lv, X., Hou, L., Li, J., Liu, Z.: Differentiating concepts and instances for knowledge graph embedding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1971–1979. Association for Computational Linguistics, Brussels, Belgium (October–November 2018). https://doi.org/10.18653/v1/D18-1222, https://aclanthology.org/D18-1222
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Ouyang, X., Yang, Y., He, L., Chen, Q., Zhang, J.: Representation learning with entity topics for knowledge graphs. In: International Conference on Knowledge Science, Engineering and Management, pp. 534–542. Springer (2017). https://doi.org/10.1007/978-3-319-63558-3_45
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Peters, M.E., et al.: Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long Papers), pp. 2227–2237. Association for Computational Linguistics, New Orleans, Louisiana, June 2018. https://doi.org/10.18653/v1/N18-1202, https://aclanthology.org/N18-1202
Qiu, H., Zheng, Q., Msahli, M., Memmi, G., Qiu, M., Lu, J.: Topological graph convolutional network-based urban traffic flow and density prediction. IEEE Trans. Intell. Transp. Syst. 22(7), 4560–4569 (2020)
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080. PMLR (2016)
Wang, D., Liu, P., Zheng, Y., Qiu, X., Huang, X.: Heterogeneous graph neural networks for extractive document summarization. arXiv preprint arXiv:2004.12393 (2020)
Wang, H., Kulkarni, V., Wang, W.Y.: DOLORES: deep contextualized knowledge graph embeddings. CoRR abs/1811.00147 (2018). http://arxiv.org/abs/1811.00147
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph and text jointly embedding. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1591–1601 (2014)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28 (2014)
Wang, Z., Li, J., Liu, Z., Tang, J.: Text-enhanced representation learning for knowledge graph. In: Proceedings of International Joint Conference on Artificial Intelligent (IJCAI), pp. 4–17 (2016)
Xie, R., Liu, Z., Luan, H., Sun, M.: Image-embodied knowledge representation learning. arXiv preprint arXiv:1609.07028 (2016)
Yang, B., Yih, W.T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014)
Yao, L., Mao, C., Luo, Y.: KG-BERT: BERT for knowledge graph completion. CoRR abs/1909.03193 (2019). http://arxiv.org/abs/1909.03193
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., Liu, Q.: ERNIE: enhanced language representation with informative entities. arXiv preprint arXiv:1905.07129 (2019)
Acknowledgements
This work was supported in part by the projects of the National Natural Science Foundation of China (61702119, 62006049), the Natural Science Foundation of Guangdong Province (2016A010101029, 2018A0303130055, 2019A1515012048), Science and Technology Program of Guangzhou (201802010029), and the Science and Technology Program of Guangzhou, China under Grant (201804010236).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, X., Ma, Z., Xiao, Z., Xia, Q., Liu, S. (2022). KGESS - A Knowledge Graph Embedding Method Based on Semantics and Structure. In: Memmi, G., Yang, B., Kong, L., Zhang, T., Qiu, M. (eds) Knowledge Science, Engineering and Management. KSEM 2022. Lecture Notes in Computer Science(), vol 13368. Springer, Cham. https://doi.org/10.1007/978-3-031-10983-6_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-10983-6_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10982-9
Online ISBN: 978-3-031-10983-6
eBook Packages: Computer ScienceComputer Science (R0)