Q-GENius: A GPT Based Modified MCQ Generator for Identifying Learner Deficiency

Abstract

This research proposes a novel methodology to address the challenge of identifying learner deficiencies from multiple choice questions (MCQs). The proposed approach involves manipulating the standard MCQ into a modified MCQ by generating distractors based on three different aspects: ‘Fact’, ‘Process’, and ‘Accuracy’. ‘Fact’ refers to the fundamental information related to the answer of the question, ‘Process’ refers to the concepts and application methods required to solve the problem, and ‘Accuracy’ refers to the degree of closeness of the answer to its true value. To achieve this, a Generative Pre-trained Transformer (GPT) model was used to develop a system called ‘Q-GENius’, that can take a question stem and some other optional additional information as input, and generate a modified MCQ with key and three distractors, each having a deficiency either in the ‘Fact,’ ‘Process,’ or ‘Accuracy’ aspect of the answer. The study was validated using inter-annotator agreement of subject matter experts on the accuracy and reproducibility of the model. The validation was performed on four different subjects. The results demonstrated that the proposed approach of modified MCQ was effective in identifying learner deficiencies when compared to standard MCQs. Overall, the proposed methodology has the potential to improve the accuracy of identifying learner deficiencies in MCQs.