Abstract
Teaching programming concepts to enhance students’ problem solving and computational thinking skills is a challenging task, especially when students enter college with little to no preparation, or they lack the interest or capacity for programming. Online platforms that serve as automated practice and assessment systems have been offered as potential tools for supporting programming skills development, providing feedback, and motivating students. The present article discusses the use of an online automated practice and assessment system called Kattis for homework assignments and final project in three computer science courses. The goal of the present study was to ascertain students’ continuance intentions to use Kattis. We attempt to address this by using partial least squares on data from a survey of 50 students. The findings of the present study suggest that continuance intentions to use Kattis is driven by students’ level of satisfaction with the system, the degree of students’ confirmation of expectations, and the perceived usefulness of the system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ala-Mutka, K. (2005). A survey of automated assessment approaches for programming assignments. Computer Science Education, 15(2), 83–102. https://doi.org/10.1080/08993400500150747.
Bazelais, P., Doleck, T., & Lemay, D. J. (2017). Investigating the predictive power of TAM: A Case Study of CEGEP Students' Intentions to Use Online Learning Technologies. Education & Information Technologies. Advance online publication. https://doi.org/10.1007/s10639-017-9587-0.
Bergin, S., & Reilly, R. (2005). The influence of motivation and comfort-level on learning to program. In Proceedings of the 17th workshop on psychology of programming (pp. 293–304). Brighton: University of Sussex.
Bhattacherjee, A. (2001). Understanding information systems continuance: An expectation-confirmation model. MIS Quarterly, 25(3), 351–370. https://doi.org/10.2307/3250921.
Blumenstein, M., Green, S., Fogelman, S., Nguyen, A., & Muthukkumarasamy, V. (2008). Performance analysis of GAME: A generic automated marking environment. Computers & Education, 50(4), 1203–1216. https://doi.org/10.1016/j.compedu.2006.11.006.
Chin, W. W. (1998). The partial least squares approach for structural equation modeling. In G. A. Marcoulides (Ed.), Modern methods for business research (pp. 295–336). Mahwah, NJ: Erlbaum.
Chiu, C., Sun, S., Sun, P., & Ju, T. (2007). An empirical analysis of the antecedents of web-based learning continuance. Computers & Education, 49(4), 1224–1245. https://doi.org/10.1016/j.compedu.2006.01.010.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Lawrence Erlbaum Associates.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.
Doleck, T., Bazelais, P., & Lemay, D. J. (2017a). Examining CEGEP students’ acceptance of CBLEs: A test of acceptance models. Education & Information Technologies, 22(5), 2523–2543. https://doi.org/10.1007/s10639-016-9559-9.
Doleck, T., Bazelais, P., & Lemay, D. J. (2017b). The role of behavioral expectations in technology Acceptance: A CEGEP Case Study. Journal of Computing in Higher Education. Advance online publication. https://doi.org/10.1007/s12528-017-9158-9.
Enström, E., Kreitz, G., Niemelä, F., Söderman, P., & Kann, V. (2011). Five years withKattis—using an automated assessment system in teaching. In Proceedings of the Frontiers in Education Conference (pp. T3J-1-T3J-6). Los Alamitos, CA: IEEE.
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363–406.
Fernandez Aleman, J. (2011). Automated assessment in a programming tools course. IEEE Transactions on Education, 54(4), 576–581. https://doi.org/10.1109/te.2010.2098442.
Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50.
Garcia-Mateos, G., & Fernandez-Aleman, J. L. (2009). Make learning fun with programming contests. In Z. Pan, A. D. Cheok, W. Müller, & A. E. Rhalibi (Eds.), Transactions on edutainment II (pp. 246–257). Berlin Heidelberg: Springer-Verlag.
Hair, J., Ringle, C., & Sarstedt, M. (2011). PLS-SEM: Indeed a silver bullet. The Journal of Marketing Theory And Practice, 19(2), 139–152. https://doi.org/10.2753/mtp1069-6679190202.
Henseler, J., Hubona, G., & Ray, P. (2016). Using PLS path modeling in new technology research: Updated guidelines. Industrial Management & Data Systems, 116(1), 2–20. https://doi.org/10.1108/imds-09-2015-0382.
Hong, S., Thong, J., & Tam, K. (2006). Understanding continued information technology usage behavior: A comparison of three models in the context of mobile internet. Decision Support Systems, 42(3), 1819–1834. https://doi.org/10.1016/j.dss.2006.03.009.
ICPC Kattis (2017). Icpc.kattis.com . Retrieved 8 July 2017, from https://icpc.kattis.com/
Jasperson, J., Carter, P. E., & Zmud, R. W. (2005). A comprehensive conceptualization of post-adoptive behaviors associated with information technology enabled work systems. MIS Quarterly, 29(3), 525–557.
Kattis (2017). Kattis.com . Retrieved 9 July 2017, From http://www.kattis.com/
Open Kattis (2017). Open.kattis.com. Retrieved 9 July 2017, from https://open.kattis.com/
Kock, N. (2015a). WarpPLS. Retrieved from http://www.warppls.com
Kock, N. (2015b). WarpPLS 5.0 user manual. ScripWarp Systems. Retrieved from http://cits.tamiu.edu/WarpPLS/UserManual_v_5_0.pdf
Layman, L., Williams, L., & Slaten, K. (2007). Note to self: Make assignments meaningful. In Proceedings of the 38th SIGCSE technical symposium on computer science education (pp. 459–463). New York: ACM.
Lee, M. (2010). Explaining and predicting users’ continuance intention toward e-learning: An extension of the expectation–confirmation model. Computers & Education, 54(2), 506–516. https://doi.org/10.1016/j.compedu.2009.09.002.
Legris, P., Ingham, J., & Collerette, P. (2003). Why do people use information technology? A critical review of the technology acceptance model. Information Management, 40(3), 191–204. https://doi.org/10.1016/s0378-7206(01)00143-4.
Lemay, D. J., Doleck, T., & Bazelais, P. (2017). “Passion and concern for privacy” as factors affecting snapchat use: A situated perspective on technology acceptance. Computers in Human Behavior, 75, 264–271. https://doi.org/10.1016/j.chb.2017.05.022.
Limayem, M., Hirt, S. G., & Cheung, C. M. K. (2007). How habit limits the predictive power of intention: The case of information systems continuance. MIS Quarterly, 31(4), 705–737.
Lin, K. (2011). E-learning continuance intention: Moderating effects of user e-learning experience. Computers & Education, 56(2), 515–526. https://doi.org/10.1016/j.compedu.2010.09.017.
Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090.
Piech, C., Huang, J., Nguyen, A., Phulsuksombati, M., Sahami, M., & Guibas, L. J. (2015). Learning program Embeddings to propagate feedback on student code. International conference on machine Learning’15, 37, 1093–1102.
Robins, A., Rountree, J., & Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education, 13(2), 137–172. https://doi.org/10.1076/csed.13.2.137.14200.
Shute, V. J. (2008). Focus on formative feedback. Source: Review of Educational Research, 78228173(1), 153–189. https://doi.org/10.3102/0034654307313795.
Stone, J., & Madigan, E. (2008). The impact of providing project choices in CS1. ACM SIGCSE Bulletin, 40(2), 65–68. https://doi.org/10.1145/1383602.1383637.
Terzis, V., Moridis, C., & Economides, A. (2013). Continuance acceptance of computer based assessment through the integration of user's expectations and perceptions. Computers & Education, 62, 50–61. https://doi.org/10.1016/j.compedu.2012.10.018.
Thong, J., Hong, S., & Tam, K. (2006). The effects of post-adoption beliefs on the expectation-confirmation model for information technology continuance. International Journal of Human-Computer Studies, 64(9), 799–810. https://doi.org/10.1016/j.ijhcs.2006.05.001.
Venkatesh, V., & Davis, F. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204. https://doi.org/10.1287/mnsc.46.2.186.11926.
Wang, T., Su, X., Ma, P., Wang, Y., & Wang, K. (2011). Ability-training-oriented automated assessment in introductory programming course. Computers & Education, 56(1), 220–226. https://doi.org/10.1016/j.compedu.2010.08.003.
Wu, C., Gerlach, J., & Young, C. (2007). An empirical analysis of open source software developers’ motivations and continuance intentions. Information Management, 44(3), 253–262. https://doi.org/10.1016/j.im.2006.12.006.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Basnet, R.B., Doleck, T., Lemay, D.J. et al. Exploring computer science students’ continuance intentions to use Kattis. Educ Inf Technol 23, 1145–1158 (2018). https://doi.org/10.1007/s10639-017-9658-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-017-9658-2