Abstract
There is much enthusiasm in higher education about the benefits of adaptive learning and using big data to investigate learning processes to make data-informed educational decisions. The benefits of adaptive learning to achieve personalized learning are obvious. Yet, there lacks evidence-based research to understand how data such as user behavior patterns can be used to design effective adaptive learning systems. The purpose of this study, therefore, is to investigate what behavior patterns learners with different characteristics demonstrate when they interact with an adaptive learning environment. Incoming 1st-year students in a pharmacy professional degree program engaged in an adaptive learning intervention that aimed to provide remedial instruction to better prepare these professional students before they began their formal degree program. We analyzed the participants’ behavior patterns through the usage data to understand how they used the adaptive system based upon their needs and interests. Using both statistical analyses and data visualization techniques, this study found: (1) apart from learners’ cognitive ability, it is important to consider affective factors such as motivation in adaptive learning, (2) lack of alignment among various components in an adaptive system can impact how learners accessed the system and, more importantly, their performance, and (3) visualizations can reveal interesting findings that can be missed otherwise. Such research should provide much needed empirical evidences and useful insights about how the analytics can inform the effective designs of adaptive learning.
Similar content being viewed by others
References
Abdous, M. H., He, W., & Yen, C. J. (2012). Using data mining for predicting relationships between online question theme and final grade. Educational Technology & Society, 15(3), 77–88.
Arroyo, I., Woolf, B. (2005) Inferring learning and attitudes from a Bayesian Network of log file data. In Proceedings of the 12th international conference on artificial intelligence in education, pp. 33–40.
Baghaei, N., Mitrovic, A., & Irwin, W. (2007). Supporting collaborative learning and problem-solving in a constraint-based CSCL environment for UML class diagrams. International Journal of Computer-Supported Collaborative Learning, 2(2), 159–190. doi:10.1007/s11412-007-9018-0.
Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief (pp. 1–57). Office of Educational Technology: US Department of Education.
Brown, J. (2015). Personalizing post-secondary education: an overview of adaptive learning solutions for higher education. Ithaka S + R. doi:10.18665/sr.221030.
Bruckman, A. (2006). Learning in online communities. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 461–472). New York: Cambridge University Press.
Brusilovsky, P. (2001). Adaptive hypermedia. User Modeling and User-Adapted Interaction, 11, 87–110. doi:10.1007/978-94-017-0617-9_1.
Brusilovsky, P., & Millán, E. (2007). User models for adaptive hypermedia and adaptive educational systems. The adaptive web: Methods and strategies of web personalization (pp. 3–53). Berlin: Springer.
Brusilovsky, P., & Vassileva, J. (2003). Course sequencing techniques for large-scale web-based education. International Journal of Continuing Engineering Education and Lifelong learning, 13(1–2), 75–94. doi:10.1504/IJCEELL.2003.002154.
Dernoncourt, F., Do, C., Halawa, S., O’Reilly, U.-M., Taylor, C., Veeramachaneni, K., & Wu, S. (2013, December). MoocViz: A large scale, open access, collaborative, data analytics platform for MOOCs. Paper presented at the NIPS Workshop on Data-Driven Education, Lake Tahoe, Nevada, USA.
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58.
Dziuban, C. D., Moskal, P. D., Cassisi, J., & Fawcett, A. (2016). Adaptive learning in psychology: Wayfinding in the digital age. Online Learning, 20(3), 74–96.
Evans, J. D. (1996). Straightforward statistics for the behavior sciences. Pacific Grove: Brooks/Cole Publishing.
Graf, S., & Liu, T. C. (2010). Analysis of learners’ navigational behavior and their learning styles in an online course. Journal of Computer Assisted learning, 26(2), 116–131. doi:10.1111/j.1365-2729.2009.00336.x.
Hsieh, T. C., Wang, T. I., Su, C. Y., & Lee, M. C. (2012). A fuzzy logic-based personalized learning system for supporting adaptive english learning. Educational Technology & Society, 15(1), 273–288. Retrieved from http://www.jstor.org/stable/jeductechsoci.15.1.273.
Hsu, P. (2012). Learner characteristic based learning effort curve mode: The core mechanism on developing personalized adaptive e-learning platform. TOJET: The Turkish Online Journal of Educational Technology, 11(4), 210–219.
Huang, S. L., & Shiu, J. H. (2012). A user-centric adaptive learning system for e-learning 2.0. Educational Technology & Society, 15(3), 214–225.
Hurley, T., & Weibelzahl, S. (2007, September). Using MotSaRT to support on-line teachers in student motivation. In Proceedings of 2nd European conference on technology enhanced learning, pp. 101–111. doi: 10.1007/978-3-540-75195-3_8.
Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016). NMC horizon report: 2016 higher education (ed.). Austin: The New Media Consortium.
Kelly, D. (2008). Adaptive Versus learner control in a multiple intelligence learning environment. Journal of educational multimedia and hypermedia, 17(3), 307–336. Retrieved from https://www.learntechlib.org/p/24252.
Kerr, P. (2016). Adaptive learning. ELT Journal, 70(1), 88–93. doi:10.1093/elt/ccv055.
Klasnja-Milicevic, A., Vesin, B., Ivanovic, M., & Budimac, Z. (2011). E-learning personalization base on hybrid recommendation strategy and learning style identification. Computers & Education, 56(3), 885–899. doi:10.1016/j.compedu.2010.11.001.
Koedinger, K., Cunningham, K., Skogsholm, A., & Leber, B. (2008). An open repository and analysis tools for fine-grained, longitudinal learner data. In International conference on educational data mining (pp. 157–166).
Liu, M., Lee, J., Kang, J., & Liu, S. (2016). What we can learn from the data: A multiple- case study examining behavior patterns by students with different characteristics in using a serious game. The Technology, Knowledge and Learning Journal, 21(1), 33–57. doi:10.1007/s10758-015-9263-7.
Liu, M., McKelroy, E., Corliss, S. B., & Carrigan, J. (2017, under review). Investigating the effect of an adaptive learning intervention on students’ learning. Submitted to Educational technology research and development.
Melero, J., Hernández-Leo, D., Sun, J., Santos, P., & Blat, J. (2015). How was the activity? A visualization support for a case of location-based learning design. British Journal of Educational Technology, 46(2), 317–329. doi:10.1111/bjet.12238.
Minović, M., Milovanović, M., Šošević, U., & González, M. Á. C. (2015). Visualisation of student learning model in serious games. Computers in Human Behavior, 47, 98–107. doi:10.1016/j.chb.2014.09.005.
Nakic, J., Granic, A., & Glavinic, V. (2015). Anatomy of student models in adaptive learning systems: A systematic literature review of individual differences from 2001 to 2013. Journal of Educational Computing Research, 51(4), 459–489. doi:10.2190/EC.51.4.e.
Papamitsiou, Z. K., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.
Premlatha, K. R., Dharani, B., & Geetha, T. V. (2014). Dynamic learner profiling and automatic learner classification for adaptive e-learning environment. Interactive Learning Environments, 24(6), 1–22. doi:10.1080/10494820.2014.948459.
Qu, H., & Chen, Q. (2015). Visual analytics for MOOC data. IEEE Computer Graphics and Applications, 35(6), 69–75. doi:10.1109/MCG.2015.137.
Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in cognitive sciences, 15(1), 20–27.
Roediger, H. L., III, & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255.
Romero, C., & Ventura, S. (2010). Educational data mining: a review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601–618. doi:10.1109/TSMCC.2010.2053532.
Sangineto, E., Capuano, N., Gaeta, M., & Micarelli, A. (2008). Adaptive course generation through learning styles representation. Universal Access in the Information Society, 7(1), 1–23. doi:10.1007/s10209-007-0101-0.
Shi, C., Fu, S., Chen, Q., & Qu, H. (2015, April). VisMOOC: Visualizing video clickstream data from massive open online courses. In 2015 IEEE Pacific visualization symposium (PacificVis) (pp. 159–166). IEEE.
Shih, B., Koedinger, K. R., & Scheines, R. (2011). A response time model for bottom-out hints as worked examples. Handbook of Educational Data Mining, 201–212.
Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400. doi:10.1177/0002764213498851.
Siemens, G., & Baker, R. S. J. D. (2012, April). Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). ACM.
Snow, E. L., Allen, L. K., Jacovina, M. E., & McNamara, D. S. (2015). Does agency matter? Exploring the impact of controlled behaviors within a game-based environment. Computers & Education, 82, 378–392. doi:10.1016/j.compedu.2014.12.011.
Van Seters, J. R., Ossevoort, M. A., Tramper, J., & Goedhart, M. J. (2012). The influence of student characteristic on the use of adaptive e-learning material. Computers & Education, 58(3), 942–952. doi:10.1016/j.compedu.2011.11.002.
Van Wijk, J. J. (2013). Evaluation: a challenge for visual analytics. Computer, 46(7), 56–60. doi:10.1109/MC.2013.151.
Wallner, G., & Kriglstein, S. (2013). Visualization-based analysis of gameplay data—A review of literature. Entertainment Computing, 4(3), 143–155. doi:10.1016/j.entcom.2013.02.002.
Yang, T., Hwang, G., & Yang, S. J. (2013). Development of an adaptive learning system with multiple perspectives based on students’ learning styles and cognitive styles. Educational Technology & Society, 16(4), 185–200.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, M., Kang, J., Zou, W. et al. Using Data to Understand How to Better Design Adaptive Learning. Tech Know Learn 22, 271–298 (2017). https://doi.org/10.1007/s10758-017-9326-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10758-017-9326-z