Fatemeh Salehian Kia
Stephanie D. Teasley
ABSTRACT
The aim of student-facing dashboards is to support learning by providing students with actionable information and promoting self-regulated learning. We created a new dashboard design aligned with SRL theory, called MyLA, to better understand how students use a learning analytics tool. We conducted sequence analysis on students' interactions with three different visualizations in the dashboard, implemented in a LMS, for a large number of students (860) in ten courses representing different disciplines. To evaluate different students' experiences with the dashboard, we computed chi-squared tests of independence on dashboard users (52%) to find frequent patterns that discriminate students by their differences in academic achievement and self-regulated learning behaviors. The results revealed discriminating patterns in dashboard use among different levels of academic achievement and self-regulated learning, particularly for low achieving students and high self-regulated learners. Our findings highlight the importance of differences in students' experience with a student-facing dashboard, and emphasize that one size does not fit all in the design of learning analytics tools.
- Rakesh. Agrawal and Ramakrishnan Srikant. 1995. Mining sequential patterns. In Proceedings of the International Conference on Data Engeneering (ICDE), In P. S. Yu and A. L. P. Chen (Eds.). IEEE Computer Society, Taipei, Taiwan, 487--499.Google Scholar
- Stephen J Aguilar. 2016. Perceived Motivational Affordances: Capturing and Measuring Students' Sense-Making Around Visualizations of their Academic Achievement Information. (2016).Google Scholar
- Sanam Shirazi Beheshitha, Marek Hatala, Dragan Gašević, and Srećko Joksimović. 2016. The role of achievement goal orientations when studying effect of learning analytics visualizations. In Proceedings of the sixth international conference on learning analytics & knowledge. ACM, 54--63.Google ScholarDigital Library
- Robert A Bjork, John Dunlosky, and Nate Kornell. 2013. Self-regulated learning: Beliefs, techniques, and illusions. Annual review of psychology 64 (2013), 417--444.Google Scholar
- Robert Bodily and Katrien Verbert. 2017. Review of research on student-facing learning analytics dashboards and educational recommender systems. IEEE Transactions on Learning Technologies (2017).Google ScholarDigital Library
- Robert Bodily and Katrien Verbert. 2017. Trends and issues in student-facing learning analytics reporting systems research. In Proceedings of the seventh international learning analytics & knowledge conference. ACM, 309--318.Google ScholarDigital Library
- Analía Cicchinelli, Eduardo Veas, Abelardo Pardo, Viktoria Pammer-Schindler, Angela Fessl, Carla Barreiros, and Stefanie Lindstädt. 2018. Finding traces of self-regulated learning in activity streams. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge. ACM, 191--200.Google ScholarDigital Library
- William S Cleveland and Robert McGill. 1984. Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American statistical association 79, 387 (1984), 531--554.Google ScholarCross Ref
- Eva Durall and Begoña Gros. 2014. Learning Analytics as a Metacognitive Tool.. In CSEDU (1). 380--384.Google Scholar
- Erik Duval. 2011. Attention please!: learning analytics for visualization and recommendation. In Proceedings of the 1st international conference on learning analytics and knowledge. ACM, 9--17.Google ScholarDigital Library
- Andrew J Elliot and Kou Murayama. 2008. On the measurement of achievement goals: Critique, illustration, and application. Journal of Educational Psychology 100, 3 (2008), 613.Google ScholarCross Ref
- Stephen Few. 2006. Information dashboard design. (2006).Google Scholar
- John Fritz. 2011. Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers. The Internet and Higher Education 14, 2 (2011), 89--97.Google ScholarCross Ref
- Alexis Gabadinho, Gilbert Ritschard, Matthias Studer, and Nicalas S Müller. 2009. Mining sequence data in R with the TraMineR package: A users guide for version 1.2. Geneva: University of Geneva (2009).Google Scholar
- Donald G Gardner, Larry L Cummings, Randall B Dunham, and Jon L Pierce. 1998. Single-item versus multiple-item measurement scales: An empirical comparison. Educational and psychological measurement 58, 6 (1998), 898--915.Google Scholar
- Dragan Gašević, Shane Dawson, and George Siemens. 2015. Let's not forget: Learning analytics are about learning. TechTrends 59, 1 (2015), 64--71.Google ScholarCross Ref
- Dragan Gasevic, Jelena Jovanovic, Abelardo Pardo, and Shane Dawson. 2017. Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics 4, 2 (2017), 113--128.Google ScholarCross Ref
- Dragan Gašević, Wannisa Matcha, Jelena Jovanović, Abelardo Pardo, Lisa-Angelique Lim, Sheridan Gentili, et al. 2019. Discovering Time Management Strategies in Learning Processes Using Process Mining Techniques. In European Conference on Technology Enhanced Learning. Springer, 555--569.Google Scholar
- Robert J Grissom and John J Kim. 2012. Effect sizes for research: Univariate and multivariate applications. Routledge.Google Scholar
- Marek Hatala and Sanam Shirazi Beheshitha. 2016. Associations Between Students ' Approaches to Learning and Learning Analytics Visualizations. In Proceedings of the LAK 2016 Workshop on Learning Analytics for Learners, Susan Bull, Blandine M. Ginon, Judy Kay, Michael D. Kickmeier-Rust, and Matthew D. Johnson (Eds.), Vol. 1596. CEUR-WS, Edinburgh, Scotland, 3--10. http://ceur-ws.org/Vol-1596/paper1.pdfGoogle Scholar
- John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research 77, 1 (2007), 81--112.Google Scholar
- Carl C Haynes, Stephanie D. Teasley, Stephanie Hayley, Meghan Oster, and John Whitmer. 2018. How am I Doing?: Student-Facing Performance Dashboards in Higher Education. In Companion Proceedings of the 8th international conference on learning analytics and knowledge. 274--275.Google Scholar
- Ioana Jivet, Maren Scheffel, Hendrik Drachsler, and Marcus Specht. 2017. Awareness is not enough: pitfalls of learning analytics dashboards in the educational practice. In European Conference on Technology Enhanced Learning. Springer, 82--96.Google ScholarCross Ref
- Ioana Jivet, Maren Scheffel, Marcus Specht, and Hendrik Drachsler. 2018. License to evaluate: preparing learning analytics dashboards for educational practice. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge. ACM, 31--40.Google ScholarDigital Library
- Jelena Jovanović, Dragan Gašević, Shane Dawson, Abelardo Pardo, and Negin Mirriahi. 2017. Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education 33, 4 (2017), 74--85.Google ScholarCross Ref
- Steven Lonn, Stephen J Aguilar, and Stephanie D Teasley. 2015. Investigating student motivation in the context of a learning analytics intervention during a summer bridge program. Computers in Human Behavior 47 (2015), 90--97.Google ScholarDigital Library
- Jock Mackinlay. 1986. Automating the design of graphical presentations of relational information. Acm Transactions On Graphics (Tog) 5, 2 (1986), 110--141.Google ScholarDigital Library
- Wannisa Matcha, Dragan Gasevic, Abelardo Pardo, et al. 2019. A Systematic Review of Empirical Studies on Learning Analytics Dashboards: A Self-Regulated Learning Perspective. IEEE Transactions on Learning Technologies (2019).Google Scholar
- Miriah Meyer, Michael Sedlmair, P Samuel Quinan, and Tamara Munzner. 2015. The nested blocks and guidelines model. Information Visualization 14, 3 (2015), 234--249.Google ScholarCross Ref
- Tamara Munzner. 2009. A nested model for visualization design and validation. IEEE transactions on visualization and computer graphics 15, 6 (2009), 921--928.Google ScholarDigital Library
- Lace M Padilla, Sarah H Creem-Regehr, Mary Hegarty, and Jeanine K Stefanucci. 2018. Decision making with visualizations: a cognitive framework across disciplines. Cognitive research: principles and implications 3, 1 (2018), 29.Google Scholar
- Steven Pinker. 1990. A theory of graph comprehension. Artificial intelligence and the future of testing (1990), 73--126.Google Scholar
- Paul R Pintrich, David AF Smith, Teresa Garcia, and Wilbert J McKeachie. 1993. Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and psychological measurement 53, 3 (1993), 801--813.Google Scholar
- Paul R Pintrich and Akane Zusho. 2002. Student motivation and self-regulated learning in the college classroom. In Higher education: Handbook of theory and research. Springer, 55--128.Google Scholar
- Gabriel Reimers, Anna Neovesky, and Akademie der Wissenschaften. 2015. Student focused dashboards. 7th International Conferenceon Computer Supported Education (Lisbon) (2015).Google Scholar
- Ido Roll and Philip H Winne. 2015. Understanding, evaluating, and supporting self-regulated learning using learning analytics. Journal of Learning Analytics 2, 1 (2015), 7--12.Google ScholarCross Ref
- Sanne FE Rovers, Geraldine Clarebout, Hans HCM Savelberg, Anique BH de Bruin, and Jeroen JG van Merriënboer. 2019. Granularity matters: comparing different ways of measuring self-regulated learning. Metacognition and Learning 14, 1 (2019), 1--19.Google ScholarCross Ref
- Richard M Ryan and Edward L Deci. 2000. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary educational psychology 25, 1 (2000), 54--67.Google Scholar
- Pierangela Samarati and Latanya Sweeney. 1998. Protecting privacy when disclosing information: k-anonymity and its enforcement through generalization and suppression. Technical Report. technical report, SRI International.Google Scholar
- Dale H Schunk and Jeffrey A Greene. 2017. Handbook of self-regulation of learning and performance. Routledge.Google Scholar
- Beat A Schwendimann, Maria Jesus Rodriguez-Triana, Andrii Vozniuk, Luis P Prieto, Mina Shirvani Boroujeni, Adrian Holzer, Denis Gillet, and Pierre Dillenbourg. 2016. Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10, 1 (2016), 30--41.Google ScholarDigital Library
- Beat A Schwendimann, Maria Jesus Rodriguez-Triana, Andrii Vozniuk, Luis P Prieto, Mina Shirvani Boroujeni, Adrian Holzer, Denis Gillet, and Pierre Dillenbourg. 2017. Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10, 1 (2017), 30--41.Google ScholarDigital Library
- Gayane Sedrakyan, Jonna Malmberg, Katrien Verbert, Sanna Järvelä, and Paul A Kirschner. 2018. Linking learning behavior analytics and learning science concepts: designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior (2018).Google Scholar
- Varshita Sher, Marek Hatala, and Dragan Gašević. 2019. On multi-device use: Using technological modality profiles to explain differences in students' learning. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge. ACM, 1--10.Google ScholarDigital Library
- Stephanie D Teasley. 2017. Student facing dashboards: One size fits all? Technology, Knowledge and Learning 22, 3 (2017), 377--384.Google ScholarCross Ref
- Katrien Verbert, Erik Duval, Joris Klerkx, Sten Govaerts, and José Luis Santos. 2013. Learning analytics dashboard applications. American Behavioral Scientist 57, 10 (2013), 1500--1509.Google ScholarCross Ref
- Katrien Verbert, Sten Govaerts, Erik Duval, Jose Luis Santos, Frans Van Assche, Gonzalo Parra, and Joris Klerkx. 2014. Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing 18, 6 (2014), 1499--1514.Google ScholarDigital Library
- Camilo Vieira, Paul Parsons, and Vetria Byrd. 2018. Visual learning analytics of educational data: A systematic literature review and research agenda. Computers & Education 122 (2018), 119--135.Google ScholarCross Ref
- Philip H Winne and Dianne Jamieson-Noel. 2002. Exploring students' calibration of self reports about study tactics and achievement. Contemporary Educational Psychology 27, 4 (2002), 551--572.Google ScholarCross Ref
- Philip H Winne and Nancy E Perry. 2000. Measuring self-regulated learning. In Handbook of self-regulation. Elsevier, 531--566.Google ScholarCross Ref
- Naomi E Winstone, Robert A Nash, Michael Parker, and James Rowntree. 2017. Supporting Learners' Agentic Engagement With Feedback: A Systematic Review and a Taxonomy of Recipience Processes. Educational Psychologist 52, 1 (2017), 17--37.Google ScholarCross Ref
- JR Young. 2016. What clicks from 70,000 courses reveal about student learning. Chronicle of Higher Education 63, 3 (2016).Google Scholar
- Mingming Zhou and Philip H Winne. 2012. Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction 22, 6 (2012), 413--419.Google ScholarCross Ref
Index Terms
- How patterns of students dashboard use are related to their achievement and self-regulatory engagement
Recommendations
Examining change in students’ self-regulated learning patterns after a formative assessment using process mining techniques
AbstractThe growing popularity of e-learning platforms, such as learning management systems, has foregrounded the role of self-regulated learning (SRL) in student success. In many e-learning environments, students typically complete learning assignments ...
Highlights- Students engage in a loosely sequenced recursive self-regulated learning cycle.
- Students actively adjust self-regulated learning patterns in response to a formative assessment.
- There are group differences in the adjustment of self-...
Promoting self-regulated learning in web-based learning environments
Self-regulated learning with the Internet or hypermedia requires not only cognitive learning strategies, but also specific and general meta-cognitive strategies. The purposes of the Study2000 project, carried out at the TU Dresden, were to develop and ...
The use of Edpuzzle to support low-achiever's development of self-regulated learning and their learning of chemistry
ICETC '18: Proceedings of the 10th International Conference on Education Technology and ComputersEducation professionals have identified that self-regulated learning skills is critical in fostering lifelong learning. Being academically at risk students, low-achievers can benefit academically if they develop self-regulated learning skills. This ...
Comments