Abstract
Systematic literature review (SLR) serves as a cornerstone of evidence-based research, yet its execution often proves time-consuming and labor-intensive. The advent of ChatGPT has sparked interest in harnessing artificial intelligence to streamline various tasks. This paper aims to evaluate the efficacy of two GPT-based tools, Elicit and SciSpace, in automating the SLR process. We assess their usability and reliability in two SLR steps: literature search and citation screening. Through testing via conducted SLR, we examine the benefits and limitations of these tools. Our findings suggest that while Elicit and SciSpace offer significant assistance in literature retrieval and study selection, their integration with human expertise is essential to ensure the thoroughness and accuracy of the review process.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agarwal, S., Laradji, I.H., Charlin, L., Pal, C.: LitLLM: A Toolkit for Scientific Literature Review. arXiv e-prints, arXiv-2402 (2024)
Al-Zubidy, A., Carver, J.C., Hale, D.P., Hassler, E.E.: Vision for SLR tooling infrastructure: prioritizing value-added requirements. Inf. Softw. Technol. 91, 72–81 (2017)
Alshami, A., Elsayed, M., Ali, E., Eltoukhy, A.E., Zayed, T.: Harnessing the power of ChatGPT for automating systematic review process: methodology, case study, limitations, and future directions. Systems 11(7), 351 (2023)
Bacinger, F., Boticki, I., Mlinaric, D.: System for semi-automated literature review based on machine learning. Electronics 11(24), 4124 (2022)
Barros Adamantino De Oliveira, B., Gerken Brasil, B.: Influence of the overweight and obesity on Systemic lupus erythematosus: A review. In incbac.org. INCBAC Institute (2023). https://www.incbac.org/wp-content/uploads/2024/04/UNIGOU-Training-Publication-2023-Bruno-Barros-Adamantino-de-Oliveira.pdf
Clark, J., McFarlane, C., Cleo, G., Ramos, C.I., Marshall, S.: The impact of systematic review automation tools on methodological quality and time taken to complete systematic review tasks: case study. JMIR Med. Educ. 7(2), e24418 (2021)
Cleo, G., Scott, A.M., Islam, F., Julien, B., Beller, E.: Usability and acceptability of four systematic review automation software packages: a mixed method design. Syst. Rev. 8, 1–5 (2019)
Gusenbauer, M., Haddaway, N.R.: Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res. Synth. Methods 11(2), 181–217 (2020)
Jain, S., Kumar, A., Roy, T., Shinde, K., Vignesh, G., Tondulkar, R.: SciSpace literature review: harnessing AI for effortless scientific discovery. In: Goharian, N., et al. Advances in Information Retrieval. ECIR 2024. LNCS, vol. 14612. Springer, Cham (2024). https://doi.org/10.1007/978-3-031-56069-9_28
Jonnalagadda, S.R., Goyal, P., Huffman, M.D.: Automating data extraction in systematic reviews: a systematic review. Syst. Rev. 4, 1–16 (2015)
Kitchenham, B., Charters, S.: Guidelines For Performing Systematic Literature Reviews in Software Engineering. Keele University (2007)
Kung, J.Y.: Elicit. J. Can. Health Libr. Assoc. J. 44(1), 15–18 (2023). https://doi.org/10.29173/jchla29657
Li, H., Scells, H., Zuccon, G.: Systematic review automation tools for end-to-end query formulation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2141–2144 (2020)
Marshall, C., Brereton, P.: Tools to support systematic literature reviews in software engineering: a mapping study. In: 2013 ACM/IEEE international symposium on empirical software engineering and measurement, pp. 296–299. IEEE (2013)
Marshall, I.J., Wallace, B.C.: Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst. Rev. 8, 1–10 (2019)
Michelson, M., Reuter, K.: The significant cost of systematic reviews and meta-analyses: a call for greater involvement of machine learning to assess the promise of clinical trials. Contemp. Clinical Trials Commun. 16, 100443 (2019)
Qureshi, R., Shaughnessy, D., Gill, K.A., Robinson, K.A., Li, T., Agai, E.: Are ChatGPT and large language models “the answer” to bringing us closer to systematic review automation? Syst. Rev. 12(1), 72 (2023)
Ray, P.P.: ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Int. Things Cyber-Phys. Syst. 3, 121–154 (2023). https://doi.org/10.1016/j.iotcps.2023.04.003
Scott, A.M., Forbes, C., Clark, J., Carter, M., Glasziou, P., Munn, Z.: Systematic review automation tools improve efficiency but lack of knowledge impedes their adoption: a survey. J. Clin. Epidemiol. 138, 80–94 (2021)
Shakeel, Y., Krüger, J., Nostitz-Wallwitz, I.V., Saake, G., Leich, T.: Automated selection and quality assessment of primary studies: a systematic literature review. J. Data Inf. Q. (JDIQ) 12(1), 1–26 (2019)
Souifi, L., Khabou, N., Rodriguez, I.B., Kacem, A.H.: Towards the use of AI-based tools for systematic literature review. ICAART 2, 595–603 (2024)
Tsafnat, G., Glasziou, P., Choong, M.K., Dunn, A., Galgani, F., Coiera, E.: Systematic review automation technologies. Syst. Rev. 3, 1–15 (2014)
Van Dinter, R., Tekinerdogan, B., Catal, C.: Automation of systematic literature reviews: a systematic literature review. Inf. Softw. Technol. 136, 106589 (2021)
Wagner, G., Lukyanenko, R., Paré, G.: Artificial intelligence and the conduct of literature reviews. J. Inf. Technol. 37(2), 209–226 (2022)
Acknowledgments
This paper is funded by the Faculty of Organizational Sciences, University of Belgrade.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Đukić, M., Škembarević, M., Jejić, O., Luković, I. (2025). Towards the Utilization of AI-Powered Assistance for Systematic Literature Review. In: Tekli, J., et al. New Trends in Database and Information Systems. ADBIS 2024. Communications in Computer and Information Science, vol 2186. Springer, Cham. https://doi.org/10.1007/978-3-031-70421-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-70421-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70420-8
Online ISBN: 978-3-031-70421-5
eBook Packages: Computer ScienceComputer Science (R0)