Abstract
Understanding cryptological primitives like encryption, hashing, signatures, and certificates is a central skill when working as an IT security professional or software developer but it is also a major educational challenge. The paper presents a study which measures and compares error rates in cryptological programming assignments. Over a ten-week period, 20 students solved 20 cryptological Java programming assignments checked by 350 tests that were automatically verified using a grader system. The error rate in ~60.000 test results is analyzed: Students made fewer errors in substitutions than transposition ciphers, symmetric ciphers rank lower than asymmetric ones, constructor, exception and padding tests appear easier to solve than signing and its verification. Asymmetric encryption has lower error rates than signing. A discussion of the findings, limitations, and possible future improvements concludes the paper. The approach allows identifying and measuring “hard” and “easy” cryptological assignments in order to improve teaching, which is desirable from an educational perspective.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Nadi, S., et al.: Jumping through hoops: why do java developers struggle with cryptography APIs? In: 2016 IEEE/ACM 38th International Conf. on Software Engineering (ICSE), pp. 935–946 (2016)
Hazhirpasand, M., et al.: Hurdles for developers in cryptography. In: 37th International Conference on Software Maintenance and Evolution (ICSME), Luxembourg (2021)
Long, F., et al.: The CERT Oracle Secure Coding Standard for Java. Addison-Wesley, Boston (2011)
Knorr, K.: Learning and grading cryptology via automated test driven software development. In: 13th IFIP WG 11.8 World Conference on Information Security Education (WISE), Maribor, Slovenia, 21–23 September, pp. 3–17 (2022)
Desai, C., Janzen, D., Savage, K.: A survey of evidence for test-driven development in academia. ACM SIGCSE Bull. 40(2), 97–101 (2008)
Edwards, S., Pérez-Quiñones, M.: Experiences using test-driven development with an automated grader. J. Comput. Sci. Coll. 22(3), 44–50 (2007)
Isong, J.: Developing an automated program checker. J. Comput. Small Coll. 16(3), 218–224 (2001)
Krusche, S., Seitz, A.: ArTEMiS - an automatic assessment management system for interactive learning. SIGCSE 2018, 21–24 February, Baltimore, MD, USA, pp. 284–289 (2018)
Braga, A., Schwab, D., Vannucci, A.: The use of acceptance test-driven development in the construction of cryptographic software. In: 9th International Conference on Emerging Security Information, Systems and Technologies (2015)
Edwards, S., et al.: Investigating static analysis errors in student java programs. In: ACM Conference on International Computing Education Research (ICER), Tacoma, pp. 65–73 (2017)
Rivers, K., Hardstead, E., Koedinger, K.: Learning curve analysis for programming: which concepts do students struggle with? In: 2016 ACM Conference on International Computing Education Research (ICER), pp. 143–151 (2016)
Sivasakthi, M., Rajendran, R.: Learning difficulties of object-oriented programming paradigm using java: students’ perspective. Indian J. Sci. Technol. 4(8), 983–985 (2011)
Lahtinen, E., Ala-Mutka, K., Järvinen, H.: A study of the difficulties of novice programmers. ACM SIGCSE Bull. 37(3), 14–18 (2005)
Lazar, D., et al.: Why does cryptographic software fail? A case study and open problems. In: 5th Asia-Pacific Workshop on Systems, pp. 1–7 (2014)
Knorr, K.: Messung der Schwierigkeit von Programmieraufgaben zur Kryptologie in Java. In: Fünfter Workshop Automatische Bewertung von Programmieraufgaben, S, pp. 35–42 (2021)
Herres, B., Oechsle, R., Schuster, D.: Der Grader ASB. In: Herausgeber Oliver, J., et al. (ed). Automatisierte Bewertung in der Programmierausbildung, Waxmann-Verlag, pp. 255–271 (2017)
Beck, K.: Test-Driven Development: By Example. Addison Wesley, Boston (2002)
JUnit Homepage. https://junit.org. Accessed 29 Jan 2022
Knorr, K.: Data files and code of the study: https://seafile.rlp.net/d/a22a20689ca1464abd79/. Accessed 27 Mar 2022
Stinson, D., Paterson, M.: Cryptography: Theory and Practice. CRC, Boca Raton (2018)
Gardner, M.: Codes, Ciphers and Secret Writing. Dover Publications, New York (1984)
Hook, D., Eaves, J.: Java Cryptography: Tools and Techniques, eBook (2022). https://leanpub.com/javacryptotoolsandtech
Weiss, J.: Java Cryptography Extensions. Morgan Kaufmann, Burlington (2004)
Bouncy Castle Homepage. https://www.bouncycastle.org. Accessed 12 Jan 2022
Rabin, M.: Digitalized Signatures and Public-Key Functions as Intractable as Factorization. MIT Laboratory for Computer Science (1979)
Vaudenay, S.: Security flaws induced by CBC padding — Applications to SSL, IPSEC, WTLS... In: Knudsen, L.R. (ed.) EUROCRYPT 2002. LNCS, vol. 2332, pp. 534–545. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46035-7_35
Effenberger, T., Čechák, J., Pelánek, R.: Measuring difficulty of introductory programming tasks. In: 6th ACM Conference on Learning@Scale, pp. 1–4 (2019)
NIST Homepage for Post-Quantum Cryptography, https://csrc.nist.gov/Projects/Post-Quantum-Cryptography. Accessed 9 Feb 2022
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Knorr, K. (2022). Analyzing Error Rates in Cryptological Programming Assignments. In: Drevin, L., Miloslavskaya, N., Leung, W.S., von Solms, S. (eds) Information Security Education - Adapting to the Fourth Industrial Revolution. WISE 2022. IFIP Advances in Information and Communication Technology, vol 650. Springer, Cham. https://doi.org/10.1007/978-3-031-08172-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-08172-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08171-2
Online ISBN: 978-3-031-08172-9
eBook Packages: Computer ScienceComputer Science (R0)