Xueyang Xing
ABSTRACT
In this paper, we propose a large language model and knowledge graph fusion reasoning method based on structured injection and causal reasoning (LKFSC) to address the limitations of existing large language models and knowledge graphs in practical applications. The approach effectively mitigates the problems of long-distance dependency and limited contextual information, and improves the reasoning capability of the large language model. Meanwhile, by fusing the generative ability of the large language model and the inference ability of the knowledge graph, the method realizes intelligent reasoning for complex problems. The main contributions of this paper include proposing a structured injection method that introduces causality for reasoning, and constructing a fusion reasoning framework that effectively mitigates the illusory problem of large language models and provides powerful and intelligent decision support for practical applications.
- Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y. (2023). ..., Tang, X., Liu, Z., Liu, P., Nie, J., & Wen, J. 2023. A Survey of Large Language Models. arXiv, abs/2303.18223.Google Scholar
- Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2021. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499.Google Scholar
- Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. 2022. Factuality enhanced language models for open-ended text generation. arXiv preprint arXiv:2206.04624.Google Scholar
- Ren, R., Wang, Y., Qu, Y., Zhao, W.X., Liu, J., Tian, H., Wu, H., Wen, J., & Wang, H. 2023. Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation. arXiv, abs/2307.11019.Google Scholar
- Chen, F., & Feng, Y. 2023. Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction.Google Scholar
- Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F. , Miller, L.E., Simens, M., Askell, A., Welinder, P., Christiano, P.F., Leike, J., & Lowe, R.J. 2022. Training language models to follow instructions with human feedback. arXiv, abs/2203.02155.Google Scholar
- Girshick, R.B., Donahue, J., Darrell, T., & Malik, J. 2013. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition,. 580-587.Google Scholar
- Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W., & Woo, W. 2015. Convolutional LSTM Network: a Machine Learning Approach for Precipitation Nowcasting. neural Information Processing Systems.Google Scholar
- Kipf, T., & Welling, M. 2016. Semi-Supervised Classification with Graph Convolutional Networks. arXiv, abs/1609.02907.Google Scholar
- Fukushima, K. 1980. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics,. 36, 193-202.Google Scholar
- Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., & Polosukhin, I. 2017. Attention is All you Need. neural Information Processing Systems.Google Scholar
- Burr, T.L. 2003. Causation, Prediction, and Search. technometrics, 45, 272 - 273.Google Scholar
- Ji, S., Pan, S., Cambria, E., Marttinen, P., & Yu, P.S. 2020. A Survey on Knowledge Graphs: Representation, Acquisition, and Applications. IEEE Transactions on Neural Networks and Learning Systems, 33, 494-514.Google ScholarCross Ref
- Zhang, J. 2023. Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT. arXiv, abs/2304.11116.Google Scholar
- Choudhary, N., & Reddy, C.K. 2023. Complex Logical Reasoning over Knowledge Graphs using Large Language Models. arXiv, abs/2305.01157.Google Scholar
- Zhu, Y., Wang, X., Chen, J., Qiao, S., Ou, Y., Yao, Y., Deng, S., Chen, H., & Zhang, N. 2023. LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities. arXiv, abs/2305.13168.Google Scholar
- Wen, Y., Wang, Z., & Sun, J. 2023. MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models. arXiv, abs/2308.09729.Google Scholar
- Shazeer, N.M. 2019. Fast Transformer Decoding: One Write-Head is All You Need. arXiv, abs/1911.02150.Google Scholar
- Jin, Zhijing, 2023 "Can Large Language Models Infer Causation from Correlation?". arXiv preprint arXiv:2306.05836Google Scholar
- Wei, Yinwei, 2022 "Causal inference for knowledge graph based recommendation." IEEE Transactions on Knowledge and Data Engineering (2022).Google Scholar
- Xu, Shuyuan, 2022 "Dynamic causal collaborative filtering." Proceedings of the 31st ACM International Conference on Information & Knowledge ManagementGoogle Scholar
- Kıcıman, Emre, 2023 "Causal reasoning and large language models: opening a new frontier for causality." arXiv preprint arXiv:2305.00050Google Scholar
- Wang, Wenjie, 2023 "Causal recommendation: progresses and future directions." Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2023.Google Scholar
- Zheng, Yu, 2021 "Disentangling user interest and conformity for recommendation with causal embedding." Proceedings of the Web Conference 2021.Google Scholar
- Chickering, David Maxwell. 2002 "Optimal structure identification with greedy search." journal of machine learning research 3. Nov: 507-554.Google Scholar
- Shimizu, Shohei, 2006 "A linear non-Gaussian acyclic model for causal discovery." Journal of Machine Learning Research 7.10.Google Scholar
- Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., Tam, W.L., Ma, Z., Xue, Y., Zhai, J., Chen, W., Zhang, P., Dong, Y., & amp; Tang, J. 2022. GLM-130B: An Open Bilingual Pre-trained Model. arXiv, abs/2210.02414.Google Scholar
- Xu, M. 2022 Text2vec: Text to vector toolkit (Version 1.1.2) [Computer software]. https://github.com/shibing624/text2vecGoogle Scholar
Index Terms
- A fusion inference method for large language models and knowledge graphs based on structured injection and causal inference
Recommendations
Hybrid knowledge fusion and inference on cloud environment
AbstractIn the age of big data, an incredible amount of knowledge is produced everywhere everyday through various ways. Knowledge fusion and inference thus has become an important issue for better utilization of knowledge. In this paper, we ...
Highlights- First to specify the knowledge in the form of association rules with temporal and spatial bindings.
MEFE: A Multi-fEature Knowledge Fusion and Evaluation Method Based on BERT
Algorithms and Architectures for Parallel ProcessingAbstractKnowledge fusion is an important part of constructing a knowledge graph. In recent years, with the development of major knowledge bases, the integration of multi-source knowledge bases is the focus and difficulty in the field of knowledge fusion. ...
Inference in multi-agent causal models
In this article, we demonstrate the usefulness of causal Bayesian networks as probabilistic reasoning systems. The biggest advantage of causal Bayesian networks over traditional probabilistic Bayesian networks is that they sometimes allow to perform ...
Comments