Authors:
Catarina Pires
1
;
Pedro Gonçalo Correia
1
;
Pedro Silva
1
and
Liliana Ferreira
1
;
2
Affiliations:
1
Faculty of Engineering, University of Porto, R. Dr. Roberto Frias, Porto, Portugal
;
2
Fraunhofer Portugal AICOS, Rua Alfredo Allen 455/461, Porto, 4200-135, Portugal
Keyword(s):
Knowledge Graphs, Large Language Models, Knowledge Graph Augmented LLMs, Academic Literature Retrieval, Natural Language Generation, Hallucination, Prompt Engineering.
Abstract:
While Large Language Models have demonstrated significant advancements in Natural Language Generation, they frequently produce erroneous or nonsensical texts. This phenomenon, known as hallucination, raises concerns about the reliability of Large Language Models, particularly when users seek accurate information, such as in academic literature retrieval. This paper addresses the challenge of hallucination in Large Language Models by integrating them with Knowledge Graphs using prompt engineering. We introduce GPTscholar, an initial study designed to enhance Large Language Models responses in the field of computer science academic literature retrieval. The authors manually evaluated the quality of responses and frequency of hallucinations on 40 prompts across 4 different use cases. We conclude that the approach is promising, as the system outperforms the results we obtained with gpt-3.5-turbo without Knowledge Graphs.