Abstract
Automatic analyses of student drawings in chemistry education have the potential to support classroom teaching. To date, related work on handwritten chemical structures or formulas is limited to well-defined presentation formats, e.g., Lewis structures. However, the large variety of possible illustrations in student drawings in chemical education has not been addressed yet. In this paper, we present a novel approach to identify visual primitives in student drawings from chemistry classes. Since the field lacks suitable datasets for the given task, we introduce a method to synthetically create a dataset for visual primitives. We demonstrate how detected visual primitives can be used to automatically classify drawings according to a taxonomy of drawing characteristics in chemistry and physics. Our experiments show that (1) the detection of visual primitives in student drawings, and (2) the subsequent classification of chemistry- and physics-specific drawing characteristics is possible.
This work has been mainly supported by the Ministry of Science and Culture of Lower Saxony, Germany, through the PhD Program “LernMINT: Data-assisted classroom teaching in the MINT subjects”.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ainsworth, S., Prain, V., Tytler, R.: Drawing to learn in science. Science 333(6046), 1096–1097 (2011). https://doi.org/10.1126/science.1204153
Bresler, M., Phan, T.V., Průša, D., Nakagawa, M., Hlavác, V.: Recognition system for on-line sketched diagrams. In: International Conference on Frontiers in Handwriting Recognition, ICFHR 2014, Crete, Greece, pp. 563–568. IEEE Computer Society (2014). https://doi.org/10.1109/ICFHR.2014.100
Bresler, M., Průša, D., Hlaváč, V.: Online recognition of sketched arrow-connected diagrams. Int. J. Doc. Anal. Recogn. (IJDAR) 19(3), 253–267 (2016). https://doi.org/10.1007/s10032-016-0269-z
Bresler, M., Průša, D., Hlavác, V.: Recognizing off-line flowcharts by reconstructing strokes and using on-line recognition techniques. In: International Conference on Frontiers in Handwriting Recognition, ICFHR 2016, Shenzhen, China, pp. 48–53. IEEE Computer Society (2016). https://doi.org/10.1109/ICFHR.2016.0022
Ha, D., Eck, D.: A neural representation of sketch drawings. In: International Conference on Learning Representations, ICLR 2018, Vancouver, Canada. OpenReview.net (2018). https://openreview.net/forum?id=Hy6GHpkCW
Hagag, A., Omara, I., Alfarra, A.N.K., Mekawy, F.: Handwritten chemical formulas classification model using deep transfer convolutional neural networks. In: International Conference on Electronic Engineering, ICEEM 2021, Menouf, Egypt, pp. 1–6 (2021). https://doi.org/10.1109/ICEEM52022.2021.9480627
Marti, U., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. Int. J. Doc. Anal. Recogn. 5(1), 39–46 (2002). https://doi.org/10.1007/s100320200071
Mouchère, H.: Online handwritten flowchart dataset (OHFCD). http://tc11.cvc.uab.es/datasets/OHFCD_1
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979). https://doi.org/10.1109/TSMC.1979.4310076
Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017). https://doi.org/10.1109/TPAMI.2016.2577031
Schäfer, B., Keuper, M., Stuckenschmidt, H.: Arrow R-CNN for handwritten diagram recognition. Int. J. Doc. Anal. Recogn. 24(1), 3–17 (2021). https://doi.org/10.1007/s10032-020-00361-1
Schäfer, B., Stuckenschmidt, H.: DiagramNet: hand-drawn diagram recognition using visual arrow-relation detection. In: Lladós, J., Lopresti, D., Uchida, S. (eds.) ICDAR 2021. LNCS, vol. 12821, pp. 614–630. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86549-8_39
Tang, K.S., Won, M., Treagust, D.: Analytical framework for student-generated drawings. Int. J. Sci. Educ. 41(16), 2296–2322 (2019). https://doi.org/10.1080/09500693.2019.1672906
Weir, H., Thompson, K., Woodward, A., Choi, B., Braun, A., Martínez, T.J.: ChemPix: automated recognition of hand-drawn hydrocarbon structures using deep learning. Chem. Sci. 12, 10622–10633 (2021). https://doi.org/10.1039/D1SC02957F
Wu, S.P.W., Rau, M.A.: How students learn content in science, technology, engineering, and mathematics (STEM) through drawing activities. Educ. Psychol. Rev. 31(1), 87–120 (2019). https://doi.org/10.1007/s10648-019-09467-3
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Stamatakis, M., Gritz, W., Oldag, J., Hoppe, A., Schanze, S., Ewerth, R. (2023). Automatic Analysis of Student Drawings in Chemistry Classes. In: Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O.C., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2023. Lecture Notes in Computer Science(), vol 13916. Springer, Cham. https://doi.org/10.1007/978-3-031-36272-9_78
Download citation
DOI: https://doi.org/10.1007/978-3-031-36272-9_78
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36271-2
Online ISBN: 978-3-031-36272-9
eBook Packages: Computer ScienceComputer Science (R0)