Abstract:
Generative AI, particularly Large Language Models (LLMs), presents innovative opportunities to enhance software engineering education. Open source LLMs such as LLaMA and ...Show MoreMetadata
Abstract:
Generative AI, particularly Large Language Models (LLMs), presents innovative opportunities to enhance software engineering education. Open source LLMs such as LLaMA and Mistral leverage the potential of generative AI offering distinct advantages over proprietary options including transparency, customizability, collaboration, and cost savings. This paper de-velops a catalog of LLM prompt examples tailored for software engineering training, mapped to knowledge areas from the Soft-ware Engineering Body of Knowledge (SWEBoK) framework. Example prompts demonstrate LLMs' capabilities in eliciting requirements, diagram generation, API simulation, effort esti-mation through role-playing, and other areas. The methodology involves evaluating prompt responses from ChatGPT, Mistral, and LLaMA on representative tasks. Quantitative and qualitative analysis assesses quality, usefulness, and correctness. Findings show ChatGPT and Mistral outperforming LLaMA overall, but no model perfectly executes complex interactions. We examine implications and challenges of integrating open source LLMs into classrooms, emphasizing the need for oversight, verification, and prompt design aligned with pedagogical objectives.
Published in: 2024 36th International Conference on Software Engineering Education and Training (CSEE&T)
Date of Conference: 29 July 2024 - 01 August 2024
Date Added to IEEE Xplore: 10 September 2024
ISBN Information: