Abstract
Automatic short answer grading (ASAG) is focusing on tackling the problem of automatically assessing students’ constructed responses to open-ended questions. ASAG is still far from being a reality in NLP. Previous work mainly concentrates on exploiting feature extraction from the textual information between the student answer and the model answer. A grade will be assigned to the student based on the similarity of his/her answers and the model answer. However, ASAG models trained by the same type of features lack the capacity to deal with a diversity of conceptual representations in students’ responses. To capture multiple types of features, prior knowledge is utilized in our work to enrich the obtained features. The whole model is based on the Transformer. More specifically, a novel training approach is proposed. Forward propagation is added in the training step randomly to exploit the textual information between the provided questions and student answers in a training step. A feature fusion layer followed by an output layer is introduced accordingly for fine-tuning purposes. We evaluate the proposed model on two datasets (the University of North Texas dataset and student response analysis (SRA) dataset). A comparison is conducted on the ASAG task between the proposed model and the baselines. The performance results show that our model is superior to the recent state-of-the-art models.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Conole, G., Warburton, B.: A review of computer-assisted assessment. Res. Learn. Technol. 13(1), 17–31 (2005)
Burrows, S., Gurevych, I., Stein, B.: The eras and trends of automatic short answer grading. Int. J. Artif. Intell. Educ. 25(1), 60–117(2015)
Roy, S., Narahari, Y., Deshmukh, O.D.: A perspective on computer assisted assessment techniques for short free-text answers. In: Ras, E., Joosten-ten Brinke, D. (eds.) CAA 2015. CCIS, vol. 571, pp. 96–109. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27704-2_10
Pulman, S.: Automarking 2: an update on the uclesoxford university research into using computational linguistics to score short free text responses (2004)
Thomas, P.: The evaluation of electronic marking of examinations. In: Proceedings of the 8th Annual Conference on Innovation and Technology in Computer Science Education, pp. 50–54 (2003). https://doi.org/10.1145/961511.961528
Pedersen, T., Patwardhan, S., WordNet, J.M.: Similarity - measuring the relatedness of concepts. In: AAAI, pp. 1024–1025. AAAI Press/The MIT Press (2004)
Mohler, M., Mihalcea, R.: Text-to-text semantic similarity for automatic short answer grading. In: EACL, pp. 567–575. The Association for Computer Linguistics (2009)
Mohler, M., Bunescu, R., Mihalcea, R.: Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In: ACL, pp. 752–762. The Association for Computer Linguistics (2011)
Dzikovska, M.O., Nielsen, R.D., Brew, C.: Semeval-2013 task 7: the joint student response analysis and 8th recognizing textual entailment challenge. In: SemEval@NAACL-HLT, pp. 263–274. The Association for Computer Linguistics (2013)
Vaswani, A., Shazeer, N., Parmar, N.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)
Claudia, L., Martin, et al.: Automated scoring of short-answer questions. Comput. Hum. 37, 92–96 (2003)
Mihalcea, R., Corley, C., Strapparava, C.: Corpus-based and knowledge-based measures of text semantic similarity. Unt Scholarly Works 1, 775–780 (2006)
Sahu, A., Bhowmick, P.K.: Feature engineering and ensemble-based approach for improving automatic short-answer grading performance. IEEE Trans. Learn. Technol. 13(1), 77–90 (2020)
Heilman, M., Madnani, N.: ETS: domain adaptation and stacking for short answer scoring. In: SemEval@NAACL-HLT, pp. 275–279. The Association for Computer Linguistics (2013)
Roy, S., Bhatt, H.S., Narahari, Y.: An iterative transfer learning based ensemble technique for automatic short answer grading. CoRR, abs/1609.04909 (2016)
Kumar, S., Chakrabarti, S., Roy, S.: Earth mover’s distance pooling over siamese LSTMs for automatic short answer grading. In: IJCAI, pp. 2046–2052. ijcai.org (2017)
Devlin, J., Chang, M.W., Lee, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (1), pp. 4171–4186. Association for Computational Linguistics (2019)
Sung, C., Dhamecha, T., Saha, S.: Pre-training BERT on domain resources for short answer grading. In: EMNLP/IJCNLP (1), pp. 6070–6074. Association for Computational Linguistics (2019)
Dai, Z., Lai, G., Yang, Y.: Funnel-transformer: filtering out sequential redundancy for efficient language processing. CoRR, abs/2006.03236 (2020)
Dzikovska, M.O., Nielsen, R., Brew, C.: Towards effective tutorial feedback for explanation questions: a dataset and baselines. In: HLT-NAACL, pp. 200–210. The Association for Computational Linguistics (2012)
Dzikovska, M.O., Isard, A., Bell, P.: BEETLE II: an adaptable tutorial dialogue system. In: SIGDIAL Conference, pp. 338–340. The Association for Computer Linguistics (2011)
Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam. CoRR, abs/1711.05101 (2017)
Liu, Y., Ott, M., Roberta, N.G.: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692 (2019)
Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: ELECTRA: pre-training text encoders as discriminators rather than generators. In: ICLR. OpenReview.net (2020)
Ramsauer, H., Schäfl, B., Lehner, J.: Hopfield networks is all you need. CoRR, abs/2008.02217 (2020)
Acknowledgement
This research was supported by NSFC (Grants No. 61877051). Li Li is the corresponding author for the paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, S., Li, L. (2021). Incorporating Question Information to Enhance the Performance of Automatic Short Answer Grading. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management. KSEM 2021. Lecture Notes in Computer Science(), vol 12817. Springer, Cham. https://doi.org/10.1007/978-3-030-82153-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-82153-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82152-4
Online ISBN: 978-3-030-82153-1
eBook Packages: Computer ScienceComputer Science (R0)