Blazenka Divjak
ABSTRACT
Learning analytics (LA) has been opening new opportunities to support learning in higher education (HE). LA dashboards are an important tool in providing students with insights into their learning progress, and predictions, leading to reflection and adaptation of learning plans and habits. Based on a human-centered approach, we present a perspective of students, as essential stakeholders, on LA dashboards. We describe a longitudinal study, based on survey methodology. The study included two iterations of a survey, conducted with second-year ICT students in 2017 (N = 222) and 2022 (N = 196). The study provided insights into the LA dashboard features the students find the most useful to support their learning. The students highly appreciated features related to short-term planning and organization of learning, while they were cautious about comparison and competition with other students, finding such features possibly demotivating. We compared the 2017 and 2022 results to establish possible changes in the students’ perspectives with the COVID-19 pandemic. The students’ awareness of the benefits of LA has increased, which may be related to the strong focus on online learning during the pandemic. Finally, a factor analysis yielded a dashboard model with five underlying factors: comparison, planning, predictions, extracurricular, and teachers.
Supplemental Material
Available for Download
docx
APPENDIX_la_dashboards_for_students.docx (17.3 KB)
APPENDIX_la_dashboards_for_students
- June Ahn, Fabio Campos, Maria Hays, and Daniela Digiacomo. 2019. Designing in Context: Reaching Beyond Usability in Learning Analytics Dashboard Design. Journal of Learning Analytics 6, 2 (July 2019). DOI:https://doi.org/10.18608/jla.2019.62.5Google Scholar
Cross Ref
- Giora Alexandron, Lisa Y. Yoo, José A. Ruipérez-Valiente, Sunbok Lee, and David E. Pritchard. 2019. Are MOOC Learning Analytics Results Trustworthy? With Fake Learners, They Might Not Be! International Journal of Artificial Intelligence in Education 29, 4 (2019), 484–506. DOI:https://doi.org/10.1007/s40593-019-00183-1Google ScholarCross Ref
- David Baneres, Ana Elena Guerrero-Roldán, M. Elena Rodríguez-González, and Abdulkadir Karadeniz. 2021. A predictive analytics infrastructure to support a trustworthy early warning system. Applied Sciences (Switzerland) 11, 13 (2021). DOI:https://doi.org/10.3390/app11135781Google ScholarCross Ref
- Liz Bennett and Sue Folley. 2020. Four design principles for learner dashboards that support student agency and empowerment. Journal of Applied Research in Higher Education 12, 1 (2020), 15–26. DOI:https://doi.org/10.1108/JARHE-11-2018-0251Google ScholarCross Ref
- Robert Bodily, Tarah K. Ikahihifo, Benjamin Mackley, and Charles R. Graham. 2018. The design, development, and implementation of student-facing learning analytics dashboards. Journal of Computing in Higher Education 30, 3 (2018), 572–598. DOI:https://doi.org/10.1007/s12528-018-9186-0Google ScholarCross Ref
- Robert Bodily and Katrien Verbert. 2017. Review of Research on Student-Facing Learning Analytics Dashboards and Educational Recommender Systems. IEEE Transactions on Learning Technologies 10, 4 (October 2017), 405–418. DOI:https://doi.org/10.1109/TLT.2017.2740172Google ScholarDigital Library
- Simon Buckingham Shum, Rebecca Ferguson, and Roberto Martinez-Maldonado. 2019. Human-Centred Learning Analytics. Journal of Learning Analytics 6, 2 (July 2019). DOI:https://doi.org/10.18608/jla.2019.62.1Google ScholarCross Ref
- Barbara M. Byrne. 2010. Structural equation modeling with AMOS: basic concepts, applications, and programming (2nd ed ed.). Routledge, New York.Google Scholar
- James Carifio and Rocco Perla. 2008. Resolving the 50-year debate around using and misusing Likert scales. Medical Education 42, 12 (December 2008), 1150–1152. DOI:https://doi.org/10.1111/j.1365-2923.2008.03172.xGoogle ScholarCross Ref
- Mohamed Amine Chatti, Arham Muslim, Manpriya Guliani, and Mouadh Guesmi. 2020. The LAVA Model: Learning Analytics Meets Visual Analytics. . 71–93. DOI:https://doi.org/10.1007/978-3-030-47392-1_5Google ScholarCross Ref
- Brynne C. DiMenichi and Elizabeth Tricomi. 2015. The power of competition: Effects of social motivation on attention, sustained physical effort, and learning. Frontiers in Psychology 6, (September 2015). DOI:https://doi.org/10.3389/fpsyg.2015.01282Google ScholarCross Ref
- Yannis Dimitriadis, Roberto Martínez-Maldonado, and Korah Wiley. 2021. Human-Centered Design Principles for Actionable Learning Analytics. In Research on E-Learning and ICT in Education. Springer International Publishing, Cham, 277–296. DOI:https://doi.org/10.1007/978-3-030-64363-8_15Google ScholarCross Ref
- Mollie Dollinger and Jason M. Lodge. 2018. Co-creation strategies for learning analytics. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge, ACM, New York, NY, USA, 97–101. DOI:https://doi.org/10.1145/3170358.3170372Google ScholarDigital Library
- Noel Entwistle. 1988. Motivation and learning strategies. Educational and Child Psychology 5, 3 (1988), 5–20.Google Scholar
- Leandre R. Fabrigar, Duane T. Wegener, Robert C. MacCallum, and Erin J. Strahan. 1999. Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods 4, 3 (September 1999), 272–299. DOI:https://doi.org/10.1037/1082-989X.4.3.272Google ScholarCross Ref
- John M. Houston, Paul B. Harris, Robert Moore, Rebecca Brummett, and Hideki Kametani. 2005. Competitiveness among Japanese, Chinese, and American Undergraduate Students. Psychological Reports 97, 1 (August 2005), 205–212. DOI:https://doi.org/10.2466/pr0.97.1.205-212Google ScholarCross Ref
- Li‐tze Hu and Peter M. Bentler. 1999. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6, 1 (January 1999), 1–55. DOI:https://doi.org/10.1080/10705519909540118Google ScholarCross Ref
- Dirk Ifenthaler and Chathuranga Widanapathirana. 2014. Development and validation of a learning analytics framework: Two case studies using support vector machines. Technology, Knowledge and Learning 19, 1–2 (2014), 221–240. DOI:https://doi.org/10.1007/s10758-014-9226-4Google ScholarCross Ref
- Ioana Jivet, Maren Scheffel, Marcel Schmitz, Stefan Robbers, Marcus Specht, and Hendrik Drachsler. 2020. From students with love: An empirical study on learner goals, self-regulated learning and sense-making of learning analytics in higher education. Internet and Higher Education 47, June (2020), 100758. DOI:https://doi.org/10.1016/j.iheduc.2020.100758Google ScholarCross Ref
- Imran Khan and Abelardo Pardo. 2016. Data2U. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge - LAK ’16, ACM Press, New York, New York, USA, 249–253. DOI:https://doi.org/10.1145/2883851.2883911Google ScholarDigital Library
- Mehmet Kokoç and Arif Altun. 2021. Effects of learner interaction with learning dashboards on academic performance in an e-learning environment. Behaviour & Information Technology 40, 2 (January 2021), 161–175. DOI:https://doi.org/10.1080/0144929X.2019.1680731Google ScholarCross Ref
- Paul Lavrakas. 2008. Encyclopedia of Survey Research Methods. Sage Publications, Inc., 2455 Teller Road, Thousand Oaks California 91320 United States of America. DOI:https://doi.org/10.4135/9781412963947Google ScholarCross Ref
- Thomas Lumley, Paula Diehr, Scott Emerson, and Lu Chen. 2002. The Importance of the Normality Assumption in Large Public Health Data Sets. Annu. Rev. Public Health 23, 1 (May 2002), 151–169. DOI:https://doi.org/10.1146/annurev.publhealth.23.100901.140546Google ScholarCross Ref
- Roberto Martinez-Maldonado, Abelardo Pardo, Negin Mirriahi, Kalina Yacef, Judy Kay, and Andrew Clayphan. 2016. LATUX: an Iterative Workflow for Designing, Validating and Deploying Learning Analytics Visualisations. Journal of Learning Analytics 2, 3 (February 2016), 9–39. DOI:https://doi.org/10.18608/jla.2015.23.3Google ScholarCross Ref
- Catherine McLoughlin and Mark J.W. Lee. 2010. Personalised and self regulated learning in the Web 2.0 era: International exemplars of innovative pedagogy using social software. Australasian Journal of Educational Technology 26, 1 (2010), 28–43. DOI:https://doi.org/10.14742/ajet.1100Google ScholarCross Ref
- Jen McPherson, Huong Ly Tong, Scott J. Fatt, and Danny Y. T. Liu. 2016. Student perspectives on data provision and use. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge - LAK ’16, ACM Press, New York, New York, USA, 158–167. DOI:https://doi.org/10.1145/2883851.2883945Google ScholarDigital Library
- Pedro J. Munoz-Merino, Manuel Fernandez Molina, Mario Munoz-Organero, and Carlos Delgado Kloos. 2014. Motivation and Emotions in Competition Systems for Education: An Empirical Study. IEEE Transactions on Education 57, 3 (August 2014), 182–187. DOI:https://doi.org/10.1109/TE.2013.2297318Google ScholarDigital Library
- Geoff Norman. 2010. Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education 15, 5 (December 2010), 625–632. DOI:https://doi.org/10.1007/s10459-010-9222-yGoogle ScholarCross Ref
- Yeonjeong Park and Il Hyun Jo. 2015. Development of the learning analytics dashboard to support students’ learning performance. Journal of Universal Computer Science 21, 1 (2015), 110–133.Google Scholar
- Paul Prinsloo and Sharon Slade. 2016. Student Vulnerability, Agency and Learning Analytics: An Exploration. Journal of Learning Analytics 3, 1 (April 2016). DOI:https://doi.org/10.18608/jla.2016.31.10Google ScholarCross Ref
- Louis M. Rea and Richard A. Parker. 2014. Designing and conducting survey research: a comprehensive guide (Fourth edition ed.). Jossey-Bass, a Wiley brand, San Francisco, CA.Google Scholar
- Alvin C. Rencher. 2002. Methods of multivariate analysis. J. Wiley.Google Scholar
- Irina Rets, Christothea Herodotou, Vaclav Bayer, Martin Hlosta, and Bart Rienties. 2021. Exploring critical factors of the perceived usefulness of a learning analytics dashboard for distance university students. International Journal of Educational Technology in Higher Education 18, 1 (2021). DOI:https://doi.org/10.1186/s41239-021-00284-9Google ScholarCross Ref
- Aaron S. Richmond, Guy A. Boysen, and Regan A. R. Gurung. 2016. An evidence-based guide to college and university teaching: developing the model teacher. Routledge, New York, NY.Google Scholar
- Lynne D. Roberts, Joel A. Howell, and Kristen Seaman. 2017. Give Me a Customizable Dashboard: Personalized Learning Analytics Dashboards in Higher Education. Technology, Knowledge and Learning 22, 3 (2017), 317–333. DOI:https://doi.org/10.1007/s10758-017-9316-1Google ScholarCross Ref
- Elizabeth B.-N. Sanders and Pieter Jan Stappers. 2008. Co-creation and the new landscapes of design. CoDesign 4, 1 (March 2008), 5–18. DOI:https://doi.org/10.1080/15710880701875068Google ScholarCross Ref
- James B. Schreiber, Frances K. Stage, Jamie King, Amaury Nora, and Elizabeth A. Barlow. 2006. Reporting structural equation modeling and confirmatory factor analysis results: A review. Journal of Educational Research 99, 6 (2006), 323–338. DOI:https://doi.org/10.3200/JOER.99.6.323-338Google ScholarCross Ref
- Clara Schumacher and Dirk Ifenthaler. 2018. Features students really expect from learning analytics. Computers in Human Behavior 78, (January 2018), 397–407. DOI:https://doi.org/10.1016/j.chb.2017.06.030Google ScholarDigital Library
- Dale H. Schunk. 2012. Learning theories: an educational perspective (6th ed ed.). Pearson, Boston.Google Scholar
- Beat A. Schwendimann, Maria Jesus Rodriguez-Triana, Andrii Vozniuk, Luis P. Prieto, Mina Shirvani Boroujeni, Adrian Holzer, Denis Gillet, and Pierre Dillenbourg. 2017. Perceiving Learning at a Glance: A Systematic Literature Review of Learning Dashboard Research. IEEE Transactions on Learning Technologies 10, 1 (January 2017), 30–41. DOI:https://doi.org/10.1109/TLT.2016.2599522Google ScholarDigital Library
- Noora Shrestha. 2021. Factor Analysis as a Tool for Survey Analysis. American Journal of Applied Mathematics and Statistics 9, 1 (January 2021), 4–11. DOI:https://doi.org/10.12691/ajams-9-1-2Google ScholarCross Ref
- Gail M. Sullivan and Anthony R. Artino. 2013. Analyzing and Interpreting Data From Likert-Type Scales. Journal of Graduate Medical Education 5, 4 (December 2013), 541–542. DOI:https://doi.org/10.4300/JGME-5-4-18Google ScholarCross Ref
- Teo Susnjak, Gomathy Suganya Ramaswami, and Anuradha Mathrani. 2022. Learning analytics dashboard: a tool for providing actionable insights to learners. International Journal of Educational Technology in Higher Education 19, 1 (December 2022), 12. DOI:https://doi.org/10.1186/s41239-021-00313-7Google ScholarCross Ref
- Bruce Thompson. 2004. Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association, Washington. DOI:https://doi.org/10.1037/10694-000Google ScholarCross Ref
- C. P. M. van der Vleuten. 1996. The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education 1, 1 (January 1996), 41–67. DOI:https://doi.org/10.1007/BF00596229Google ScholarCross Ref
- Cees P M van der Vleuten and Lambert W T Schuwirth. 2005. Assessing professional competence: from methods to programmes. Medical Education 39, 3 (March 2005), 309–317. DOI:https://doi.org/10.1111/j.1365-2929.2005.02094.xGoogle ScholarCross Ref
- J. F.C. de Winter and D. Dodou. Five-Point Likert Items: t test versus Mann-Whitney-Wilcoxon (Addendum added October 2012). DOI:https://doi.org/10.7275/BJ1P-TS64Google ScholarCross Ref
Index Terms
- Learning analytics dashboards: What do students actually ask for?
Index terms have been assigned to the content through auto-classification.
Recommendations
How Do Teachers Use Dashboards Enhanced with Data Storytelling Elements According to their Data Visualisation Literacy Skills?
LAK2023: LAK23: 13th International Learning Analytics and Knowledge ConferenceThere is a proliferation of learning analytics (LA) dashboards aimed at supporting teachers. Yet, teachers still find it challenging to make sense of LA dashboards, thereby making informed decisions. Two main strategies to address this are emerging: i) ...
Team interactions with learning analytics dashboards
AbstractStudent-facing visualisations have attracted increased attention with recent developments in data-driven tools to support individual and group work. Learning analytics dashboards (LADs), a data-enhanced feedback tool that allows ...
Highlights- Learning analytics dashboard in K-12 collaborative learning promotes engagement and collaboration.
Features students really expect from learning analytics
More and more learning in higher education settings is being facilitated through online learning environments. Students' ability to self-regulate their learning is considered a key factor for success in higher education. Learning analytics offer a ...
Comments