Skip to main content
Springer Nature Link
Log in

Analysing the Role of Generative AI in Software Engineering - Results from an MLR

  • Conference paper
  • First Online:
Systems, Software and Services Process Improvement (EuroSPI 2024)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2179))

Included in the following conference series:

  • 1085 Accesses

Abstract

Generative Artificial Intelligence (GenAI) has become a practical tool that exhibits the potential to revolutionize numerous industries through publicly available systems with simple yet effective interfaces. This paper outlines the findings of research conducted in a multivocal literature review (MLR) with the aim of exploring the impact of GenAI in software engineering, with a focus on the fundamental aspects, use cases, benefits, and risks associated with contemporary GenAI models leveraged in key industries and practices. Key findings indicate that GenAI is adopted in software engineering, with various reported benefits in areas including requirement engineering, estimation and testing. However, there are also some risks associated with GenAI-based Software Engineering, such as in the context of generated data consistency and accuracy (sometimes referred to as the Hallucination problem), plagiarism, bias, and security. GenAI-assisted software engineering is becoming more mainstream, but resolving all the associated issues is going to take some time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Brühl, V.: Generative Artificial Intelligence – Foundations, Use Cases and Economic Potential. Intereconomics 59(1), 5–9 Walter de Gruyter GmbH (2024). https://doi.org/10.2478/ie-2024-0003

  2. Kumar, S., Musharaf, D., Musharaf, S., Sagar, A.K.: A Comprehensive Review of the Latest Advancements in Large Generative AI Models. Communications in Computer and Information Science. Springer Nature Switzerland, pp. 90–103. (2023). https://doi.org/10.1007/978-3-031-45121-8_9

  3. Feuerriegel, S., Hartmann, J., Janiesch, C., et al.: Generative AI. Bus Inf. Syst. Eng. 66, 111–126 (2024). https://doi.org/10.1007/s12599-023-00834-7

    Article  Google Scholar 

  4. Tomczak, J.M.: Deep generative modeling. Springer International Publishing (2022). https://doi.org/10.1007/978-3-030-93158-2

    Article  Google Scholar 

  5. Alfasly, S., Nejat, P., Hemati, S., et al.: When is a Foundation Model a Foundation Model. https://arxiv.org/pdf/2309.11510.pdf

  6. Jung, K.H.: Uncover This Tech Term: Foundation Model. Korean J. Radiol. 24(10), 1038–1041 (2023). https://doi.org/10.3348/kjr.2023.0790.PMID:37793672;PMCID:PMC10550749

    Article  Google Scholar 

  7. Moore, C., Taylor, T., Anderson, C.: Exploring the Frontiers of Generative AI: From ChatGPT to Multimodal Innovations. https://www.researchgate.net/profile/Charles-Anderson-32/publication/376831367_Exploring_the_Frontiers_of_Generative_AI_From_ChatGPT_to_Multimodal_Innovations/links/658b469a3c472d2e8e90733a/Exploring-the-Frontiers-of-Generative-AI-From-ChatGPT-to-Multimodal-Innovations.pdf

  8. Banh, L., Strobel, G.: Generative artificial intelligence. Electron Markets 33, 63 (2023). https://doi.org/10.1007/s12525-023-00680-1

    Article  Google Scholar 

  9. Janiesch, C., Zschech, P., Heinrich, K.: Machine learning and deep learning. Electron Markets 31, 685–695 (2021). https://doi.org/10.1007/s12525-021-00475-2

    Article  Google Scholar 

  10. Sagar, S., Hongyi, Z., Balaji, P., Yidong, C., Hsinchun, C., Nunamaker Jr, J.F.: Deep Learning for Information Systems Research. J. Manag. Inf. Syst. 40(1), 271–301 (2023). https://doi.org/10.1080/07421222.2023.2172772

    Article  Google Scholar 

  11. Cao, Y., Li, S., Liu, Y.: A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT. https://arxiv.org/pdf/2303.04226.pdf

  12. Naveed, H., Ullah Khan, A., Qiu, S.: A Comprehensive Overview of Large Language Models. https://arxiv.org/pdf/2307.06435.pdf

  13. Kalyan, K.S., Rajasekharan, A., Sangeetha, S.: AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing. https://arxiv.org/pdf/2108.05542.pdf

  14. Ouyang, L., Wu, J., Jiang, X., et al.: Training language models to follow instructions with human feedback. https://proceedings.neurips.cc/paper_files/paper/2022/file/b1efde53be364a73914f58805a001731-Paper-Conference.pdf

  15. Sorin, V., Klang, E.: Large language models and the emergence phenomena. European Journal of Radiology Open. 10, 100494 (2023). ISSN 2352-0477. https://doi.org/10.1016/j.ejro.2023.100494

  16. Chui, M., et al.: The Economic Potential of Generative AI: The Next Productivity Frontier. McKinsey & Company, June 14, 2023. https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier

  17. Generative AI: From Buzz to Business Value. Accessed February 23, 2024. https://kpmg.com/kpmg-us/content/dam/kpmg/pdf/2023/generative-ai-survey.pdf

  18. Pothukuchi, A.S., Lakshmi Vasuda, K., Mallikarjunaradhya, V.: Impact of Generative AI on the Software Development Lifecycle (SDLC). SSRN, August 22, 2023. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4536700

  19. Dohmke, T.: The Economic Impact of the AI-Powered Developer Lifecycle and Lessons from Github Copilot. The GitHub Blog, (2023). https://github.blog/2023-06-27-the-economic-impact-of-the-ai-powered-developer-lifecycle-and-lessons-from-github-copilot/

  20. Ozkaya, I.: Application of Large Language Models to Software Engineering Tasks: Opportunities, Risks, and Implications. IEEE Softw. 40(3), 4–8, (2023). https://doi.org/10.1109/MS.2023.3248401

  21. Robert, J.E., Schmidt, D.: (Vanderbilt University). Generative AI Q&A: Applications in Software Engineering. SEI Blog, November 16, 2023. https://insights.sei.cmu.edu/blog/generative-ai-question-and-answer-applications-in-software-engineering/

  22. Islam, M., Khan, F., Alam, S., Hasan, M.: Artificial Intelligence in Software Testing: A Systematic Review. TENCON 2023 - 2023 IEEE Region 10 Conference (TENCON), Chiang Mai, Thailand, pp. 524–529 (2023). https://doi.org/10.1109/TENCON58879.2023.10322349

  23. Layman, L., Vetter, R.: Generative Artificial Intelligence and the Future of Software Testing. Computer 57(1), 27–32 (2024). https://doi.org/10.1109/MC.2023.3306998

  24. Ebert, C., Louridas, P.: Generative AI for Software Practitioners. IEEE Softw. 40(4), 30–38 (2023). https://doi.org/10.1109/MS.2023.3265877

  25. OpenAI: “Introducing Gpts.” November 6, 2023, Accessed: February 23, 2024. https://openai.com/blog/introducing-gpts

  26. AWS: “What is prompt engineering? - ai prompt engineering explained”, Accessed February 23, 2024. https://aws.amazon.com/what-is/prompt-engineering/

  27. Schmidt, D.C., Spencer-Smith, J., Fu, Q., White, J.: Towards a Catalog of Prompt Patterns to Enhance the Discipline of Prompt Engineering. https://www.dre.vanderbilt.edu/~schmidt/PDF/ADA-User-Journal.pdf

  28. Search Results for ‘Prompt Engineer’ Jobs. Indeed. Accessed: Feb. 22, 2024. https://www.indeed.com/q-prompt-engineer-jobs.html

  29. Arora, C., Grundy, J., Abdelrazek, M.: Advancing Requirements Engineering through Generative AI: Assessing the Role of LLMs. https://arxiv.org/pdf/2310.13976.pdf

  30. Fu, M., Tantithamthavorn, C.: GPT2SP: A Transformer-Based Agile Story Point Estimation Approach. IEEE Trans. Softw. Eng. 49(2), 611–625, (2023). https://doi.org/10.1109/TSE.2022.3158252

  31. Dohmke, T.: Sea Change in Software Development: Economic and Productivity Analysis of the AI-Powered Developer Lifecycle. https://arxiv.org/ftp/arxiv/papers/2306/2306.15033.pdf

  32. Peng, S., Kalliamvakou, E., Cihon, P., Demirer, M.: The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. https://arxiv.org/pdf/2302.06590.pdf

  33. A Culturally Sensitive Test to Evaluate Nuanced GPT Hallucination. IEEE J. Mag. IEEE Xplore. ieeexplore.ieee.org. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10319443 (accessed Feb. 23, 2024)

  34. Smith, C.: ChatGPT’s Hallucinations Could Keep It from Succeeding - IEEE Spectrum. spectrum.ieee.org (2023). https://spectrum.ieee.org/ai-hallucination

  35. Ando, K., Okumura, T., Komachi, M., Horiguchi, H., Matsumoto, Y.: Is artificial intelligence capable of generating hospital discharge summaries from inpatient records? PLOS Digital Health 1(12), e0000158 (2022). https://doi.org/10.1371/journal.pdig.0000158

    Article  Google Scholar 

  36. Velásquez-Henao, J.D., Franco-Cardona, C.J., Cadavid-Higuita, L.: Prompt Engineering: a methodology for optimizing interactions with AI-Language Models in the field of engineering. https://revistas.unal.edu.co/index.php/dyna/article/view/111700/90275

  37. Ray, P.P.: ChatGPT: a Comprehensive Review on background, applications, Key challenges, bias, ethics, Limitations and Future Scope. Internet of Things and Cyber-Phys. Syst. 3(1), 121–154 (2023). https://doi.org/10.1016/j.iotcps.2023.04.003

    Article  Google Scholar 

  38. Perry, N., Srivastava, M., Kumar, D., Boneh, D.: Do Users Write More Insecure Code with AI Assistants? arXiv (Cornell University) (2022). https://doi.org/10.1145/3576915.3623157

  39. Application of Large Language Models to Software Engineering Tasks: Opportunities, Risks, and Implications, IEEE Journals & Magazine, IEEE Xplore. ieeexplore.ieee.org. https://ieeexplore.ieee.org/abstract/document/10109345 (accessed Feb. 23, 2024) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10109345

  40. Dien, J.: Editorial: Generative Artificial Intelligence as a Plagiarism Problem. p. 108621, (2023) https://doi.org/10.1016/j.biopsycho.2023.108621

  41. Kirova, V.D., Ku, C.S., Laracy, J.R., Marlowe, T.J.: The Ethics of Artificial Intelligence in the Era of Generative AI. J. Syst. Cybern. Inform. 21(4), 42–50 (2023) https://doi.org/10.54808/jsci.21.04.42

  42. Vanian, J., Leswing, K.: ChatGPT and Generative AI are booming, but at a very expensive price. CNBC. (2023) https://www.cnbc.com/2023/03/13/chatgpt-and-generative-ai-are-booming-but-at-a-very-expensive-price.html

  43. Freed, M., et al.: An Investigation of Green Software Engineering. In: Proceedings of Systems, Software and Services Process Improvement, EuroSPI 2023, 29–31 August 2022, Grenoble, France. Communications in Computer and Information Science (CCIS), vol 1881. Springer, Cham (2022)

    Google Scholar 

  44. De Buitlear, C., et al.: Investigating Sources and Effects of Bias in AI-Based Systems – Results from an MLR. In: Proceedings of Systems, Software and Services Process Improvement, EuroSPI 2023, 29–31 August 2022, Grenoble, France. Communications in Computer and Information Science (CCIS), vol 1881. Springer, Cham (2022)

    Google Scholar 

  45. Dagg, N., et al.: Socially-Critical Software Systems: Is Extended Regulation Required? In: Proceedings of Systems, Software and Services Process Improvement, EuroSPI 2022, 30 August 2022 - 2 September 2022, Salzburg, Austria. Communications in Computer and Information Science (CCIS), vol 1646. Springer, Cham. pp. 610–622. (2022) https://doi.org/10.1007/978-3-031-15559-8_43

  46. Meade E., et al.: The Changing Role of the Software Engineer. In: Proceedings of the 26th European and Asian Conference on Systems, Software and Services Process Improvement (EuroSPI 2019), Springer CCIS Vol. 1060, pp.682–694, 18–20. Edinburgh, Scotland (2019)

    Google Scholar 

  47. Clarke, P., O'Connor, R.V.: Changing situational contexts present a constant challenge to software developers. In: Proceedings of the 22nd European and Asian Conference on Systems, Software and Services Process Improvement (EuroSPI 2015), CCIS (Vol. 543), pp. 100–111, 30 September - 02 October 2015, Ankara, Turkey (2015)

    Google Scholar 

  48. Clarke, P., O'Connor, R.V.: An Approach to Evaluating Software Process Adaptation, In: Proceedings of the 11th International Conference on Software Process Improvement and Capability dEtermination (SPICE 2011), CCIS Vol. 155, pp. 28–41. Heidelberg, Germany: Springer-Verlag (2011)

    Google Scholar 

  49. Marks, G., O'Connor, R.V., Clarke, P.: The impact of situational context on the software development process - A case study of a highly innovative start-up organization. In: Proceedings of the 17th International SPICE Conference (SPICE 2017), pp. 455–466; 4–5. Palma de Mallorca, Spain (2017)

    Google Scholar 

  50. Clarke, P.: The Remote Working Genie Is Out of the Office Bottle. IEEE Software. 40(4), 88–95 (2023). https://doi.org/10.1109/MS.2023.3258921

Download references

Acknowledgements

This research is supported in part by SFI, Science Foundation Ireland (https://www.sfi.ie/) grant No SFI 13/RC/2094_P2 to Lero - the Science Foundation Ireland Research Centre for Software.

Author information

Authors and Affiliations

  1. School of Computing, Dublin City University, Dublin, Ireland

    Tuomas Bazzan, Benjamin Olojo, Przemysław Majda, Thomas Kelly & Paul M. Clarke

  2. Department of Computer Engineering, Gazi University, Ankara, Turkey

    Murat Yilmaz

  3. Faculty of Engineering and Computing, Dublin City University, Dublin, Ireland

    Gerard Marks

  4. Lero the Science Foundation Ireland Research Centre for Software, Limerick, Ireland

    Paul M. Clarke

Authors
  1. Tuomas Bazzan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin Olojo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Przemysław Majda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Murat Yilmaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gerard Marks
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Paul M. Clarke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul M. Clarke .

Editor information

Editors and Affiliations

  1. Gazi University, Ankara, Türkiye

    Murat Yilmaz

  2. Dublin City University, Dublin, Ireland

    Paul Clarke

  3. Grenoble Alpes University, Grenoble, France

    Andreas Riel

  4. I.S.C.N. GesmbH, Graz, Austria

    Richard Messnarz

  5. Munich University of Applied Sciences, Munich, Germany

    Christian Greiner

  6. Munich University of Applied Sciences, Munich, Germany

    Thomas Peisl

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bazzan, T. et al. (2024). Analysing the Role of Generative AI in Software Engineering - Results from an MLR. In: Yilmaz, M., Clarke, P., Riel, A., Messnarz, R., Greiner, C., Peisl, T. (eds) Systems, Software and Services Process Improvement. EuroSPI 2024. Communications in Computer and Information Science, vol 2179. Springer, Cham. https://doi.org/10.1007/978-3-031-71139-8_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-71139-8_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-71138-1

  • Online ISBN: 978-3-031-71139-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics