Loading [a11y]/accessibility-menu.js
Estimating the Difficulty of Programming Problems Using Fine-tuned LLM | IEEE Conference Publication | IEEE Xplore

Estimating the Difficulty of Programming Problems Using Fine-tuned LLM


Abstract:

Currently, many competitive programming contests have been held. These contests are composed of several problems of a wide range of difficulty levels. Novice programmers ...Show More

Abstract:

Currently, many competitive programming contests have been held. These contests are composed of several problems of a wide range of difficulty levels. Novice programmers must approach problems based on their experience and levels to avoid losing confidence and motivation to learn programming skills. In some programming contests, the difficulty level of a problem is given as a numerical value. However, each contest site assigns difficulty levels based on its criteria, making comparing levels among two or more contest sites impossible. This study proposes a method to estimate the difficulty level from the problem description and example solution information. Specifically, we attempted to estimate the difficulty of the problems using GPT-3.5 Turbo, which was fine-tuned with the problem descriptions and example solutions. The experiment was conducted on the fine-tuned GPT-3.5 Turbo under five conditions to evaluate the performance of difficulty estimation. As a result, a model that was fine-tuned with problem descriptions and estimated difficulty from the problem description had the best performance. In addition, the performance of the model fine-tuned with problem descriptions only was better than that of the model fine-tuned with problem descriptions and example solutions.
Date of Conference: 30 May 2024 - 01 June 2024
Date Added to IEEE Xplore: 26 September 2024
ISBN Information:

ISSN Information:

Conference Location: Honolulu, HI, USA

Contact IEEE to Subscribe

References

References is not available for this document.