Abstract
In this work, we address automatic evaluation of the difficulty of programming problems or exercises. Typically the problems consist of both text description and accompanying figures. We collect a suitable dataset, investigate the evaluation based on the text and the image data separately, as well as a combination of the two. The first results of this investigation are reported, together with the discussion and future work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bollapragada, R., Nocedal, J., Mudigere, D., Shi, H.J., Tang, P.T.P.: A progressive batching L-BFGS method for machine learning. In: International Conference on Machine Learning, pp. 620–629. PMLR (2018)
Chen, H., Lagadec, B., Bremond, F.: Partition and reunion: a two-branch neural network for vehicle re-identification. In: CVPR Workshops, pp. 184–192 (2019)
Galar, M., Fernández, A., Barrenechea, E., Bustince, H., Herrera, F.: An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recogn. 44(8), 1761–1776 (2011)
Hoeting, J.A., Madigan, D., Raftery, A.E., Volinsky, C.T.: Bayesian model averaging: a tutorial. Stat. Sci. 382–401 (1999)
Huang, Z., et al.: Question difficulty prediction for reading problems in standard tests. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Ke, G., et al.: LightGBM: a highly efficient gradient boosting decision tree. Adv. Neural. Inf. Process. Syst. 30, 3146–3154 (2017)
Koutsojannis, C., Beligiannis, G., Hatzilygeroudis, I., Papavlasopoulos, C., Prentzas, J.: Using a hybrid AI approach for exercise difficulty level adaptation. Int. J. Continuing Eng. Educ. Life Long Learn. 17(4–5), 256–272 (2007)
Myles, A.J., Feudale, R.N., Liu, Y., Woody, N.A., Brown, S.D.: An introduction to decision tree modeling. J. Chemometr.: J. Chemometr. Soc. 18(6), 275–285 (2004)
O’Shea, K., Nash, R.: An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458 (2015)
Palmer, D.D.: Text preprocessing. Handb. Nat. Lang. Process. 2, 9–30 (2010)
Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, Heidelberg (2010). https://doi.org/10.1007/978-1-84882-935-0
Zhou, Z.H.: Ensemble learning. Encycl. Biometr. 1, 270–273 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Skarbalius, A., Lukoševičius, M. (2021). Automatic Programming Problem Difficulty Evaluation – First Results. In: Lopata, A., Gudonienė, D., Butkienė, R. (eds) Information and Software Technologies. ICIST 2021. Communications in Computer and Information Science, vol 1486. Springer, Cham. https://doi.org/10.1007/978-3-030-88304-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-88304-1_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88303-4
Online ISBN: 978-3-030-88304-1
eBook Packages: Computer ScienceComputer Science (R0)