Abstract
The learning assessment seeks to collect data that allows for identifying learning gaps for teacher decision-making. Hence, teachers need to plan and select various assessment instruments that enable the verification of learning evolution. Considering that a more significant number of evaluation instruments and modalities increase the teachers’ workload, the adoption of machine learning might support the assessing actions and amplify the potential of students’ observation and follow-up. This article aims to analyze machine learning algorithms for automatic classification of free-text answers, i.e., evaluating descriptive questions written in Portuguese. We utilized a dataset of 9981 free-text answers for 17 questions. After pre-processing the data, we used eight classification algorithms. In conclusion, we highlight that the Logistic Regression, ExtraTrees, Random Forest, and Multi-layer Perceptron algorithms obtained results above 0.9 of F-score for both multi-class and binary classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
dos Santos, J.C.A., Ribeiro, T., Favero, E., Queiroz, J.: Aplicação de um método lsa na avaliação automática de respostas discursivas. In: Anais do Workshop de Desafios da Computação Aplicada à Educação, pp. 10–19 (2012)
Passero, G., Haendchen Filho, A., Dazzi, R.: Avaliaçao do uso de métodos baseados em lsa e wordnet para correçao de questoes discursivas. In: Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), vol. 27, p. 1136 (2016)
Çınar, A., Ince, E., Gezer, M., Yılmaz, Ö.: Machine learning algorithm for grading open-ended physics questions in Turkish. Educ. Inf. Technol. 25(5), 3821–3844 (2020)
Basu, S., Jacobs, C., Vanderwende, L.: Powergrading: a clustering approach to amplify human effort for short answer grading. Trans. Assoc. Comput. Ling. 1, 391–402 (2013)
Zehner, F., Sälzer, C., Goldhammer, F.: Automatic coding of short text responses via clustering in educational assessment. Educ. Psychol. Measur. 76(2), 280–303 (2016)
Horbach, A., Pinkal, M.: Semi-supervised clustering for short answer scoring. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) (2018)
Leacock, C., Chodorow, M.: C-rater: automated scoring of short-answer questions. Comput. Humanit. 37(4), 389–405 (2003)
Willis, A.: Using NLP to support scalable assessment of short free text responses. In: Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 243–253 (2015)
Marvaniya, S., Saha, S., Dhamecha, T.I., Foltz, P., Sindhgatta, R., Sengupta, B.: Creating scoring rubric from representative student answers for improved short answer grading. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 993–1002 (2018)
Sahu, A., Bhowmick, P.K.: Feature engineering and ensemble-based approach for improving automatic short-answer grading performance. IEEE Trans. Learn. Technol. 13(1), 77–90 (2019)
Vij, S., Tayal, D., Jain, A.: A machine learning approach for automated evaluation of short answers using text similarity based on wordnet graphs. Wireless Pers. Commun. 111(2), 1271–1282 (2020)
Filighera, A., Steuer, T., Rensing, C.: Fooling automatic short answer grading systems. In: Bittencourt, I.I., Cukurova, M., Muldner, K., Luckin, R., Millán, E. (eds.) AIED 2020. LNCS (LNAI), vol. 12163, pp. 177–190. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52237-7_15
Galhardi, L., Barbosa, C.R., de Souza, R.C.T., Brancher, J.D.: Portuguese automatic short answer grading. In: Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), vol. 29, p. 1373 (2018)
Liu, T., Ding, W., Wang, Z., Tang, J., Huang, G.Y., Liu, Z.: Automatic short answer grading via multiway attention networks. In: Isotani, S., Millán, E., Ogan, A., Hastings, P., McLaren, B., Luckin, R. (eds.) AIED 2019. LNCS (LNAI), vol. 11626, pp. 169–173. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23207-8_32
Lun, J., Zhu, J., Tang, Y., Yang, M.: Multiple data augmentation strategies for improving performance on automatic short answer scoring. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 13389–13396 (2020)
Tan, H., Wang, C., Duan, Q., Lu, Y., Zhang, H., Li, R.: Automatic short answer grading by encoding student responses via a graph convolutional network. In: Interactive Learning Environments, pp. 1–15 (2020)
Zhang, Y., Lin, C., Chi, M.: Going deeper: automatic short-answer grading by combining student and question models. User Model. User-Adap. Inter. 30(1), 51–80 (2020)
Kapoor, B.S.J., Nagpure, S.M., Kolhatkar, S.S., Chanore, P.G., Vishwakarma, M.M., Kokate, R.B.: An analysis of automated answer evaluation systems based on machine learning. In: 2020 International Conference on Inventive Computation Technologies (ICICT), pp. 439–443. IEEE (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Gomes Rocha, F., Rodriguez, G., Andrade, E.E.F., Guimarães, A., Gonçalves, V., Sabino, R.F. (2021). Supervised Machine Learning for Automatic Assessment of Free-Text Answers. In: Batyrshin, I., Gelbukh, A., Sidorov, G. (eds) Advances in Soft Computing. MICAI 2021. Lecture Notes in Computer Science(), vol 13068. Springer, Cham. https://doi.org/10.1007/978-3-030-89820-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-89820-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89819-9
Online ISBN: 978-3-030-89820-5
eBook Packages: Computer ScienceComputer Science (R0)