Zhenzhou Lin
ABSTRACT
Knowledge graph embedding aims at modeling knowledge by projecting entities and relations into a low-dimensional semantic space. Most of the works on knowledge graph embedding construct negative samples by negative sampling as knowledge graphs typically only contain positive facts. Although substantial progress has been made by dynamic distribution based sampling methods, selecting plausible and prior information-engaged negative samples still poses many challenges. Inspired by type constraint methods, we propose Hierarchical Type Enhanced Negative Sampling (HTENS) which leverages hierarchical entity type information and entity-relation cooccurrence information to optimize the sampling probability distribution of negative samples. The experiments performed on the link prediction task demonstrate the effectiveness of HTENS. Additionally, HTENS shows its superiority in versatility and can be integrated into scalable systems with enhanced negative sampling.
Supplemental Material
- Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data (Vancouver, Canada) (SIGMOD '08). ACM, New York, NY, USA, 1247--1250. https://doi.org/10.1145/1376616.1376746Google Scholar
Digital Library
- Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems, Vol. 26. Curran Associates, Inc., 2787--2795. https://proceedings.neurips.cc/paper/2013/file/ 1cecc7a77928ca8133fa24680a88d2f9-Paper.pdfGoogle Scholar
- Liwei Cai and William Yang Wang. 2018. KBGAN: Adversarial Learning for Knowledge Graph Embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Long Papers), Vol. 1. ACL, New Orleans, Louisiana, 1470--1480. https://doi.org/10.18653/v1/N18-1133Google ScholarCross Ref
- Chenhe Dong, Yinghui Li, Haifan Gong, Miaoxin Chen, Junxin Li, Ying Shen, and Min Yang. 2022. A Survey of Natural Language Generation. ACM Comput. Surv. 55, 8, Article 173 (dec 2022), 38 pages. https://doi.org/10.1145/3554727Google ScholarDigital Library
- Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong, and Qing He. 2022. A Survey on Knowledge Graph-Based Recommender Systems. IEEE Transactions on Knowledge and Data Engineering 34, 08 (aug 2022), 3549--3568. https://doi.org/10.1109/TKDE.2020.3028705Google ScholarCross Ref
- Xu Han, Shulin Cao, Xin Lv, Yankai Lin, Zhiyuan Liu, Maosong Sun, and Juanzi Li. 2018. OpenKE: An Open Toolkit for Knowledge Embedding. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. ACL, Brussels, Belgium, 139--144. https://doi.org/10.18653/v1/ D18--2024Google ScholarCross Ref
- Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge Graph Embedding via Dynamic Mapping Matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). ACL, Beijing, China, 687--696. https://doi.org/10.3115/v1/P15-1067Google ScholarCross Ref
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations. San Diego, CA, USA.Google Scholar
- Denis Krompaß, Stephan Baier, and Volker Tresp. 2015. Type-Constrained Representation Learning in Knowledge Graphs. In Proceedings of the 14th International Semantic Web Conference, Vol. 9366. Springer, Bethlehem, PA, USA, 640--655.Google ScholarDigital Library
- Yunshi Lan, Gaole He, Jinhao Jiang, Jing Jiang, Wayne Xin Zhao, and Ji-Rong Wen. 2021. A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Zhi-Hua Zhou (Ed.). International Joint Conferences on Artificial Intelligence Organization, Montreal, 4483--4491. https://doi.org/10.24963/ijcai.2021/611 Survey Track.Google ScholarCross Ref
- Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality. ACL, Beijing, China, 57--66. https://doi.org/10.18653/v1/W15-4007Google ScholarCross Ref
- Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex Embeddings for Simple Link Prediction. In Proceedings of The 33rd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 48). PMLR, New York, New York, USA, 2071--2080. https: //proceedings.mlr.press/v48/trouillon16.htmlGoogle Scholar
- Peifeng Wang, Shuangyin Li, and Rong Pan. 2018. Incorporating GAN for Negative Sampling in Knowledge Representation Learning. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI'18). AAAI Press, New Orleans, Louisiana, USA, 2005--2012.Google ScholarCross Ref
- Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge Graph Embedding by Translating on Hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Vol. 28. AAAI Press, Québec City, Québec, Canada, 1112--1119.Google ScholarCross Ref
- Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2016. Representation Learning of Knowledge Graphs with Hierarchical Types. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. AAAI Press, New York, NY, USA, 2965--2971.Google ScholarDigital Library
- Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In 3rd International Conference on Learning Representations. San Diego, CA, USA. http://arxiv.org/abs/1412.6575Google Scholar
- Yongqi Zhang, Quanming Yao, Yingxia Shao, and Lei Chen. 2019. NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding. In 35th IEEE International Conference on Data Engineering. IEEE, Macao, China, 614--625. https://doi.org/10.1109/ICDE.2019.00061Google ScholarCross Ref
Index Terms
- Hierarchical Type Enhanced Negative Sampling for Knowledge Graph Embedding
Recommendations
Entity-Relation Distribution-Aware Negative Sampling for Knowledge Graph Embedding
The Semantic Web – ISWC 2023AbstractKnowledge Graph Embedding (KGE) is a powerful technique for mining knowledge from knowledge graphs. Negative sampling plays a critical role in KGE training and significantly impacts the performance of KGE models. Negative sampling methods ...
Entity Similarity-Based Negative Sampling for Knowledge Graph Embedding
PRICAI 2022: Trends in Artificial IntelligenceAbstractKnowledge graph embedding (KGE) models optimize loss functions to maximize the total plausibility of positive triples and minimize the plausibility of negative triples. Negative samples are essential in KGE training since they are not as ...
Simple and automated negative sampling for knowledge graph embedding
AbstractNegative sampling, which samples negative triplets from non-observed ones in knowledge graph (KG), is an essential step in KG embedding. Recently, generative adversarial network (GAN) has been introduced in negative sampling. By sampling negative ...
Comments