Skip to main content

Towards the Utilization of AI-Powered Assistance for Systematic Literature Review

  • Conference paper
  • First Online:
New Trends in Database and Information Systems (ADBIS 2024)

Abstract

Systematic literature review (SLR) serves as a cornerstone of evidence-based research, yet its execution often proves time-consuming and labor-intensive. The advent of ChatGPT has sparked interest in harnessing artificial intelligence to streamline various tasks. This paper aims to evaluate the efficacy of two GPT-based tools, Elicit and SciSpace, in automating the SLR process. We assess their usability and reliability in two SLR steps: literature search and citation screening. Through testing via conducted SLR, we examine the benefits and limitations of these tools. Our findings suggest that while Elicit and SciSpace offer significant assistance in literature retrieval and study selection, their integration with human expertise is essential to ensure the thoroughness and accuracy of the review process.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agarwal, S., Laradji, I.H., Charlin, L., Pal, C.: LitLLM: A Toolkit for Scientific Literature Review. arXiv e-prints, arXiv-2402 (2024)

    Google Scholar 

  2. Al-Zubidy, A., Carver, J.C., Hale, D.P., Hassler, E.E.: Vision for SLR tooling infrastructure: prioritizing value-added requirements. Inf. Softw. Technol. 91, 72–81 (2017)

    Article  Google Scholar 

  3. Alshami, A., Elsayed, M., Ali, E., Eltoukhy, A.E., Zayed, T.: Harnessing the power of ChatGPT for automating systematic review process: methodology, case study, limitations, and future directions. Systems 11(7), 351 (2023)

    Article  Google Scholar 

  4. Bacinger, F., Boticki, I., Mlinaric, D.: System for semi-automated literature review based on machine learning. Electronics 11(24), 4124 (2022)

    Article  Google Scholar 

  5. Barros Adamantino De Oliveira, B., Gerken Brasil, B.: Influence of the overweight and obesity on Systemic lupus erythematosus: A review. In incbac.org. INCBAC Institute (2023). https://www.incbac.org/wp-content/uploads/2024/04/UNIGOU-Training-Publication-2023-Bruno-Barros-Adamantino-de-Oliveira.pdf

  6. Clark, J., McFarlane, C., Cleo, G., Ramos, C.I., Marshall, S.: The impact of systematic review automation tools on methodological quality and time taken to complete systematic review tasks: case study. JMIR Med. Educ. 7(2), e24418 (2021)

    Article  Google Scholar 

  7. Cleo, G., Scott, A.M., Islam, F., Julien, B., Beller, E.: Usability and acceptability of four systematic review automation software packages: a mixed method design. Syst. Rev. 8, 1–5 (2019)

    Article  Google Scholar 

  8. Gusenbauer, M., Haddaway, N.R.: Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res. Synth. Methods 11(2), 181–217 (2020)

    Article  Google Scholar 

  9. Jain, S., Kumar, A., Roy, T., Shinde, K., Vignesh, G., Tondulkar, R.: SciSpace literature review: harnessing AI for effortless scientific discovery. In: Goharian, N., et al. Advances in Information Retrieval. ECIR 2024. LNCS, vol. 14612. Springer, Cham (2024). https://doi.org/10.1007/978-3-031-56069-9_28

  10. Jonnalagadda, S.R., Goyal, P., Huffman, M.D.: Automating data extraction in systematic reviews: a systematic review. Syst. Rev. 4, 1–16 (2015)

    Article  Google Scholar 

  11. Kitchenham, B., Charters, S.: Guidelines For Performing Systematic Literature Reviews in Software Engineering. Keele University (2007)

    Google Scholar 

  12. Kung, J.Y.: Elicit. J. Can. Health Libr. Assoc. J. 44(1), 15–18 (2023). https://doi.org/10.29173/jchla29657

  13. Li, H., Scells, H., Zuccon, G.: Systematic review automation tools for end-to-end query formulation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2141–2144 (2020)

    Google Scholar 

  14. Marshall, C., Brereton, P.: Tools to support systematic literature reviews in software engineering: a mapping study. In: 2013 ACM/IEEE international symposium on empirical software engineering and measurement, pp. 296–299. IEEE (2013)

    Google Scholar 

  15. Marshall, I.J., Wallace, B.C.: Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst. Rev. 8, 1–10 (2019)

    Article  Google Scholar 

  16. Michelson, M., Reuter, K.: The significant cost of systematic reviews and meta-analyses: a call for greater involvement of machine learning to assess the promise of clinical trials. Contemp. Clinical Trials Commun. 16, 100443 (2019)

    Article  Google Scholar 

  17. Qureshi, R., Shaughnessy, D., Gill, K.A., Robinson, K.A., Li, T., Agai, E.: Are ChatGPT and large language models “the answer” to bringing us closer to systematic review automation? Syst. Rev. 12(1), 72 (2023)

    Article  Google Scholar 

  18. Ray, P.P.: ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Int. Things Cyber-Phys. Syst. 3, 121–154 (2023). https://doi.org/10.1016/j.iotcps.2023.04.003

    Article  Google Scholar 

  19. Scott, A.M., Forbes, C., Clark, J., Carter, M., Glasziou, P., Munn, Z.: Systematic review automation tools improve efficiency but lack of knowledge impedes their adoption: a survey. J. Clin. Epidemiol. 138, 80–94 (2021)

    Article  Google Scholar 

  20. Shakeel, Y., Krüger, J., Nostitz-Wallwitz, I.V., Saake, G., Leich, T.: Automated selection and quality assessment of primary studies: a systematic literature review. J. Data Inf. Q. (JDIQ) 12(1), 1–26 (2019)

    Google Scholar 

  21. Souifi, L., Khabou, N., Rodriguez, I.B., Kacem, A.H.: Towards the use of AI-based tools for systematic literature review. ICAART 2, 595–603 (2024)

    Google Scholar 

  22. Tsafnat, G., Glasziou, P., Choong, M.K., Dunn, A., Galgani, F., Coiera, E.: Systematic review automation technologies. Syst. Rev. 3, 1–15 (2014)

    Article  Google Scholar 

  23. Van Dinter, R., Tekinerdogan, B., Catal, C.: Automation of systematic literature reviews: a systematic literature review. Inf. Softw. Technol. 136, 106589 (2021)

    Article  Google Scholar 

  24. Wagner, G., Lukyanenko, R., Paré, G.: Artificial intelligence and the conduct of literature reviews. J. Inf. Technol. 37(2), 209–226 (2022)

    Article  Google Scholar 

Download references

Acknowledgments

This paper is funded by the Faculty of Organizational Sciences, University of Belgrade.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Marija Đukić .

Editor information

Editors and Affiliations

Ethics declarations

Disclosure of Interests

The authors have no competing interests to declare that are relevant to the content of this article.

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Đukić, M., Škembarević, M., Jejić, O., Luković, I. (2025). Towards the Utilization of AI-Powered Assistance for Systematic Literature Review. In: Tekli, J., et al. New Trends in Database and Information Systems. ADBIS 2024. Communications in Computer and Information Science, vol 2186. Springer, Cham. https://doi.org/10.1007/978-3-031-70421-5_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-70421-5_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-70420-8

  • Online ISBN: 978-3-031-70421-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics