Loading [a11y]/accessibility-menu.js
Leveraging Open Source LLMs for Software Engineering Education and Training | IEEE Conference Publication | IEEE Xplore

Leveraging Open Source LLMs for Software Engineering Education and Training


Abstract:

Generative AI, particularly Large Language Models (LLMs), presents innovative opportunities to enhance software engineering education. Open source LLMs such as LLaMA and ...Show More

Abstract:

Generative AI, particularly Large Language Models (LLMs), presents innovative opportunities to enhance software engineering education. Open source LLMs such as LLaMA and Mistral leverage the potential of generative AI offering distinct advantages over proprietary options including transparency, customizability, collaboration, and cost savings. This paper de-velops a catalog of LLM prompt examples tailored for software engineering training, mapped to knowledge areas from the Soft-ware Engineering Body of Knowledge (SWEBoK) framework. Example prompts demonstrate LLMs' capabilities in eliciting requirements, diagram generation, API simulation, effort esti-mation through role-playing, and other areas. The methodology involves evaluating prompt responses from ChatGPT, Mistral, and LLaMA on representative tasks. Quantitative and qualitative analysis assesses quality, usefulness, and correctness. Findings show ChatGPT and Mistral outperforming LLaMA overall, but no model perfectly executes complex interactions. We examine implications and challenges of integrating open source LLMs into classrooms, emphasizing the need for oversight, verification, and prompt design aligned with pedagogical objectives.
Date of Conference: 29 July 2024 - 01 August 2024
Date Added to IEEE Xplore: 10 September 2024
ISBN Information:

ISSN Information:

Conference Location: Würzburg, Germany

References

References is not available for this document.