Zhen Tan
ABSTRACT
Knowledge graphs are typical large-scale multi-relational structures, which comprise a large amount of fact triplets. Nonetheless, existing knowledge graphs are still sparse and far from being complete. To refine the knowledge graphs, representation learning is widely used to embed fact triplets into low-dimensional spaces. Many existing knowledge graph embedding models either focus on learning rich features from entities but fail to extract good features of relations, or employ sophisticated models that have rather high time and memory-space complexities. In this paper, we propose a novel knowledge graph embedding model, CombinE. It exploits entity features from two complementary perspectives via the plus and minus combinations. We start with the plus combination, where we use shared features of entity pairs participating in a relation to convey its relation features. To also allow differences of each pairs of entities participating in a relation, we also use the minus combination, where we concentrate on individual entity features, and regard relations as a channel to offset the divergence and preserve the prominence between head and tail entities. Compared with the state-of-the-art models, our experimental results demonstrate that CombinE outperforms existing ones and has low time and memory-space complexities.
- A. Bordes, J. Weston, R. Collobert, and Y. Bengio. Learning structured embeddings of knowledge bases. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, San Francisco, California, USA, August 7--11, 2011, 2011. Google Scholar
Cross Ref
- A. Bordes, X. Glorot, J. Weston, and Y. Bengio. Joint learning of words and meaning representations for open-text semantic parsing. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, La Palma, Canary Islands, April 21--23, 2012, pages 127--135, 2012.Google Scholar
- -Durán, Weston, and Yakhnenko}DBLP:conf/nips/BordesUGWY13A. Bordes, N. Usunier, A. García-Durán, J. Weston, and O. Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5--8, 2013, Lake Tahoe, Nevada, United States., pages 2787--2795, 2013. Google ScholarDigital Library
- A. Bordes, X. Glorot, J. Weston, and Y. Bengio. A semantic matching energy function for learning with multi-relational data - application to word-sense disambiguation. Machine Learning, 94 (2): 233--259, 2014. Google ScholarDigital Library
- N. Chapados, Y. Bengio, P. Vincent, J. Ghosn, C. Dugas, I. Takeuchi, and L. Meng. Estimating car insurance premia: a case study in high-dimensional data inference. In Advances in Neural Information Processing Systems 14 {Neural Information Processing Systems: Natural and Synthetic, December 3--8, 2001, Vancouver, British Columbia, Canada}, pages 1369--1376, 2001. Google ScholarDigital Library
- X. Dong, E. Gabrilovich, G. Heitz, W. Horn, N. Lao, K. Murphy, T. Strohmann, S. Sun, and W. Zhang. Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA - August 24 - 27, 2014, pages 601--610, 2014. Google ScholarDigital Library
- J. Feng, M. Huang, M. Wang, M. Zhou, Y. Hao, and X. Zhu. Knowledge graph embedding by flexible translation. In Principles of Knowledge Representation and Reasoning: Proceedings of the Fifteenth International Conference, Cape Town, South Africa, April 25--29, 2016., pages 557--560, 2016. Google ScholarDigital Library
- T. Franz, A. Schultz, S. Sizov, and S. Staab. Triplerank: Ranking semantic web data by tensor decomposition. In The Semantic Web - ISWC 2009, 8th International Semantic Web Conference, Chantilly, VA, USA, October 25--29, 2009. Proceedings, pages 213--228, 2009. Google ScholarDigital Library
- X. Glorot, A. Bordes, and Y. Bengio. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, April 11--13, 2011, pages 315--323, 2011.Google Scholar
- R. Jenatton, N. L. Roux, A. Bordes, and G. Obozinski. A latent factor model for highly multi-relational data. In Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3--6, 2012, Lake Tahoe, Nevada, United States., pages 3176--3184, 2012. Google ScholarDigital Library
- G. Ji, S. He, L. Xu, K. Liu, and J. Zhao. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, July 26--31, 2015, Beijing, China, Volume 1: Long Papers, pages 687--696, 2015.Google Scholar
- Y. Jia, Y. Wang, H. Lin, X. Jin, and X. Cheng. Locally adaptive translation for knowledge graph embedding. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12--17, 2016, Phoenix, Arizona, USA., pages 992--998, 2016. Google ScholarDigital Library
- C. Kemp, J. B. Tenenbaum, T. L. Griffiths, T. Yamada, and N. Ueda. Learning systems of concepts with an infinite relational model. In Proceedings, The Twenty-First National Conference on Artificial Intelligence and the Eighteenth Innovative Applications of Artificial Intelligence Conference, July 16--20, 2006, Boston, Massachusetts, USA, pages 381--388, 2006. Google ScholarDigital Library
- Y. Lin, Z. Liu, M. Sun, Y. Liu, and X. Zhu. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25--30, 2015, Austin, Texas, USA., pages 2181--2187, 2015. Google ScholarDigital Library
- M. Nickel, V. Tresp, and H. Kriegel. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on Machine Learning, Bellevue, Washington, USA, June 28 - July 2, 2011, pages 809--816, 2011. Google ScholarDigital Library
- M. Nickel, L. Rosasco, and T. A. Poggio. Holographic embeddings of knowledge graphs. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12--17, 2016, Phoenix, Arizona, USA., pages 1955--1961, 2016. Google ScholarDigital Library
- M. Richardson and P. M. Domingos. Markov logic networks. Machine Learning, 62 (1--2): 107--136, 2006. Google ScholarDigital Library
- P. Singla and P. M. Domingos. Entity resolution with markov logic. In Proceedings of the 6th IEEE International Conference on Data Mining, 18--22 December 2006, Hong Kong, China, pages 572--582, 2006. Google ScholarDigital Library
- R. Socher, D. Chen, C. D. Manning, and A. Y. Ng. Reasoning with neural tensor networks for knowledge base completion. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5--8, 2013, Lake Tahoe, Nevada, United States., pages 926--934, 2013. Google ScholarDigital Library
- J. Sun, D. Tao, and C. Faloutsos. Beyond streams and graphs: dynamic tensor analysis. In Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, PA, USA, August 20--23, 2006, pages 374--383, 2006. Google ScholarDigital Library
- I. Sutskever, R. Salakhutdinov, and J. B. Tenenbaum. Modelling relational data using bayesian clustered tensor factorization. In Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7--10 December 2009, Vancouver, British Columbia, Canada., pages 1821--1828, 2009. Google ScholarDigital Library
- T. Trouillon, J. Welbl, S. Riedel, É. Gaussier, and G. Bouchard. Complex embeddings for simple link prediction. In Proceedings of the 33nd International Conference on Machine Learning, New York City, NY, USA, June 19--24, 2016, pages 2071--2080, 2016. Google ScholarDigital Library
- Z. Wang, J. Zhang, J. Feng, and Z. Chen. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, July 27-31, 2014, Québec City, Québec, Canada., pages 1112--1119, 2014. Google ScholarDigital Library
- R. Xie, Z. Liu, J. Jia, H. Luan, and M. Sun. Representation learning of knowledge graphs with entity descriptions. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12--17, 2016, Phoenix, Arizona, USA., pages 2659--2665, 2016. Google ScholarDigital Library
- B. Yang, W. Yih, X. He, J. Gao, and L. Deng. Embedding entities and relations for learning and inference in knowledge bases. CoRR, abs/1412.6575, 2014.Google Scholar
- M. D. Zeiler. ADADELTA: an adaptive learning rate method. CoRR, abs/1212.5701, 2012.Google Scholar
Index Terms
- Representation Learning of Large-Scale Knowledge Graphs via Entity Feature Combinations
Recommendations
Duet Representation Learning with Entity Multi-attribute Information in Knowledge Graphs
Advanced Data Mining and ApplicationsAbstractRepresentation Learning (RL) of knowledge graphs aims to project both entities and relations into a continuous low-dimension space. Most methods concentrate on learning entities’ representations with structure information indicating the relations ...
Facing Changes: Continual Entity Alignment for Growing Knowledge Graphs
The Semantic Web – ISWC 2022AbstractEntity alignment is a basic and vital technique in knowledge graph (KG) integration. Over the years, research on entity alignment has resided on the assumption that KGs are static, which neglects the nature of growth of real-world KGs. As KGs grow,...
Learning Knowledge Representation with Entity Concept Information
Advanced Data Mining and ApplicationsAbstractThe goal of Knowledge Representation Learning (KRL) is to learn an accurate knowledge representation that conforms to human understanding. Currently, many works have used entity multi-source information to improve the entity representation ...
Comments