Skip to main content
SpringerLink
Log in

Knowledge Fusion via Joint Tensor and Matrix Factorization

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

We consider the task of knowledge fusion, an important aspect of cognitive intelligence, with the goal of combining part-of knowledge drawn from different sources. For this, entities and relations are cast into matrix-based representations. Unlike previous work on relation prediction, we consider the challenging setting of graphs with large amounts of completely separate connected components and no overlap between the training and test set entities. In order to address these challenges, we propose a novel cognitively inspired factorization method that jointly factorizes a subject–predicate–object tensor via RESCAL and a similarity matrix via matrix factorization. Our experimental results show that our method significantly outperforms several strong baseline models, including RESCAL and several TransE-style models. The proposed joint factorization of a subject–predicate–object tensor while applying matrix factorization to a similarity matrix obtains substantially higher average accuracy rates than previous approaches. This shows that it can successfully address the challenge of knowledge fusion of disconnected data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://www.geonames.org/

  2. https://www.numpy.org/

  3. https://github.com/thunlp/KB2E

References

  1. Acar E, Rasmussen M, Savorani F, Næs T., Bro R. 2013. Understanding data fusion within the framework of coupled matrix and tensor factorizations 129, 53-63.

  2. Bordes A, Glorot X, Weston J, Bengio Y. 2013. A semantic matching energy function for learning with multi-relational data. Machine Learning. To appear.

  3. Bordes A, Usunier N, García-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. NIPS; 2013. p. 2787–2795.

  4. Boyd SP, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning 2011;3(1): 1–122.

    Google Scholar 

  5. Chew PA, Bader BW, Kolda TG, Abdelali A. Cross-language information retrieval using PARAFAC2. SIGKDD; 2007. p. 143–152.

  6. Correa FE, Oliveira MDB, Gama J, Corrêa PLP, Rady J. Analyzing the behavior dynamics of grain price indexes using tucker tensor decomposition and spatio-temporal trajectories. Comput Electron Agric 2016; 120:72-78.

    Google Scholar 

  7. Dong X, Gabrilovich E, Heitz G, Horn W, Lao N, Murphy K, Strohmann T, Sun S, Zhang W. Knowledge vault: a web-scale approach to probabilistic knowledge fusion. SIGKDD; 2014. p. 601–610.

  8. Dong XL, Gabrilovich E, Heitz G, Horn W, Murphy K, Sun S, Zhang W. From data fusion to knowledge fusion. PVLDB 2014;7(10):881–892.

    Google Scholar 

  9. Dong XL, Srivastava D. Knowledge curation and knowledge fusion: challenges, models and applications. Proceedings of the 2015 ACM SIGMOD international conference on management of data, Melbourne, Victoria, Australia, May 31 - June 4, 2015; 2015. p. 2063–2066.

  10. Du J, Yuan C, Tian P, Lin H. Channel estimation for multi-input multi-output relay systems using the PARATUCK2 tensor model. IET Commun 2016;10(9):995–1002.

    Google Scholar 

  11. Fellbaum C, (ed). 1998. WordNet: an electronic lexical database. http://www.amazon.ca/exec/obidos/redirect?tag=citeulike04-20%&path=ASIN/026206197X. Cambridge: The MIT Press.

  12. He L, Liu B, Li G, Sheng Y, Wang Y, Xu Z. Knowledge base completion by variational Bayesian neural tensor decomposition. Cogn Comput 2018;10(6):1075–1084.

    Google Scholar 

  13. He S, Liu K, Ji G, Zhao J. Learning to represent knowledge graphs with gaussian embedding. CIKM. ACM; 2015. p. 623–632.

  14. Ji G, Liu K, He S, Zhao J. Knowledge graph completion with adaptive sparse transfer matrix. AAAI; 2016. p. 985–991.

  15. Jiang JJ, Conrath DW. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. arXiv:cmp-lg/9709008.

  16. Kiers H. Towards a standardized notation and terminology in multiway analysis. 2000;14, 105-122.

  17. Kolda TG, Bader BW. Tensor decompositions and applications. SIAM Rev 2009;51(3):455–500.

    Google Scholar 

  18. Krishna R, Zhu Y, Groth O, Johnson J, Hata K, Kravitz J, Chen S, Kalantidis Y, Li LJ, Shamma DA, Bernstein M, Fei-Fei L. 2016. Visual genome: connecting language and vision using crowdsourced dense image annotations. arXiv:1602.07332 .

  19. Leacock C, Chodorow M. Combining local context and wordnet similarity for word sense identification. WordNet: an electronic lexical database 1998;49(2):265–283.

    Google Scholar 

  20. Lin D. An information-theoretic definition of similarity. ICML; 1998.

  21. Lin Y, Liu Z, Luan H, Sun M, Rao S, Liu S. 2015. Modeling relation paths for representation learning of knowledge bases. Computer Science.

  22. Lin Y, Liu Z, Sun M, Liu Y, Zhu X. Learning entity and relation embeddings for knowledge graph completion. AAAI; 2015. p. 2181–2187.

  23. Lin Y, Liu Z, Zhu X, Zhu X, Zhu X. Learning entity and relation embeddings for knowledge graph completion. AAAI; 2015. p. 2181–2187.

  24. Liu B, He L, Li Y, Zhe S, Xu Z. Neuralcp: Bayesian multiway data analysis with neural tensor decomposition. Cogn Comput 2018;10(6):1051–1061.

    Google Scholar 

  25. Mikolov T, Chen K, Corrado G, Dean J. 2013. Efficient estimation of word representations in vector space. arXiv:1301.3781.

  26. Nengfu X, Wensheng W, Xiaorong Y, Lihua J. Rule-based agricultural knowledge fusion in web information integration. NJAS - Wageningen Journal of Life Sciences 2012;10(1):635–638(4).

    Google Scholar 

  27. Nickel M, Tresp V, Kriegel H. A three-way model for collective learning on multi-relational data. ICML; 2011. p. 809–816.

  28. Nickel M, Tresp V, Kriegel H. Factorizing YAGO: scalable machine learning for linked data. WWW; 2012. p. 271–280.

  29. Pennington J, Socher R, Manning CD. Glove: Global vectors for word representation. EMNLP; 2014. p. 1532–1543.

  30. Pilehvar MT, Jurgens D, Navigli R. Align, disambiguate and walk: a unified approach for measuring semantic similarity. ACL; 2013. p. 1341–1351.

  31. Preece AD, Hui K, Gray WA, Marti P, Bench-Capon TJM, Jones DM, Cui Z. The KRAFT architecture for knowledge fusion and transformation. Knowl.-Based Syst 2000;13(2-3):113–120.

    Google Scholar 

  32. Ragusa E, Gastaldo P, Zunino R, Cambria E. Learning with similarity functions: a tensor-based framework. Cogn Comput 2019;11(1):31–49.

    Google Scholar 

  33. Resnik P. 1995. Using information content to evaluate semantic similarity in a taxonomy. arXiv:cmp-lg/9511007.

  34. Solé-Casals J, Caiafa CF, Zhao Q, Cichocki AS. Brain-computer interface with corrupted EEG data: a tensor completion approach. Cogn Comput 2018;10(6):1062–1074.

    Google Scholar 

  35. Stegeman A, Berge JT, Psychometrika LDL. Sufficient conditions for uniqueness in candecomp/parafac and indscal with random component matrices. Psychometrika 2006;71(2):219–229.

    PubMed  Google Scholar 

  36. Tandon N, Hariman C, Urbani J, Rohrbach A, Rohrbach M, Weikum G. Commonsense in parts: mining part-whole relations from the web and image tags. Proceedings of the thirtieth AAAI conference on artificial intelligence, February 12–17, 2016, Phoenix, Arizona, USA; 2016. p. 243–250.

  37. Thoma S, Rettinger A, Both F. 2017. Knowledge fusion via embeddings from text, knowledge graphs, and images. arXiv:1704.06084.

  38. Wang Y, Widrow B, Zadeh LA, Howard N, Wood S, Bhavsar VC, Budin G, Chan CW, Fiorini RA, Gavrilova ML, Shell DF. Cognitive intelligence: deep learning, thinking, and reasoning by brain-inspired systems. IJCINI 2016;10(4):1–20.

    Google Scholar 

  39. Wang Z, Zhang J, Feng J, Chen Z. Knowledge graph embedding by translating on hyperplanes. AAAI; 2014. p. 1112–1119.

  40. Wu Z, Palmer MS. Verb semantics and lexical selection. ACL; 1994. p. 133–138.

  41. Yu Xl, Qiao L. 2017. Knowledge fusion methods: a survey. DEStech Transactions on Computer Science and Engineering (smce).

  42. Zhang J, Han Y, Jiang J. Tucker decomposition-based tensor learning for human action recognition. Multimedia Syst 2016;22(3):343–353.

    Google Scholar 

Download references

Funding

The authors would like to acknowledge the financial support provided by the National Natural Science Foundation of China (no. 61503217)

Author information

Authors and Affiliations

  1. Shandong University, Jinan, 250101, China

    Zengguang Hao, Yafang Wang & Zining Liu

  2. Rutgers University, New York, NY, 10002, USA

    Gerard de Melo

  3. University of Electronic Science and Technology of China, Chengdu, China

    Zenglin Xu

Authors
  1. Zengguang Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yafang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zining Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gerard de Melo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zenglin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yafang Wang.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not describe any studies with human participants or animals performed by any of the authors.

Informed Consent

Informed consent was not required as no humans or animals were involved.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hao, Z., Wang, Y., Liu, Z. et al. Knowledge Fusion via Joint Tensor and Matrix Factorization. Cogn Comput 12, 642–653 (2020). https://doi.org/10.1007/s12559-019-09686-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-019-09686-4

Keywords

Search

Navigation