skip to main content
10.1145/2883851.2883914acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
research-article

A pedagogical framework for learning analytics in collaborative inquiry tasks: an example from a teamwork competency awareness program

Published: 25 April 2016 Publication History

Abstract

Many pedagogical models in the field of learning analytics are implicit and do not overtly direct learner behavior. While this allows flexibility of use, this could also result in misaligned practice, and there are calls for more explicit pedagogical models in learning analytics. This paper presents an explicit pedagogical model, the Team and Self Diagnostic Learning (TSDL) framework, in the context of collaborative inquiry tasks. Key informing theories include experiential learning, collaborative learning, and the learning analytics process model. The framework was trialed through a teamwork competency awareness program for 14 year old students. A total of 272 students participated in the program. This paper foregrounds students' and teachers' evaluative accounts of the program. Findings reveal positive perceptions of the stages of the TSDL framework, despite identified challenges, which points to its potential usefulness for teaching and learning. The TSDL framework aims to provide theoretical clarity of the learning process, and foster alignment between learning analytics and the learning design. The current work provides trial outcomes of a teamwork competency awareness program that used dispositional analytics, and further efforts are underway to develop the discourse layer of the analytic engine. Future work will also be dedicated to application and refinement of the framework for other contexts and participants, both learners and teachers alike.

References

[1]
Ali, L., Hatala, M., Gašević, D., and Jovanović, J., 2012. A qualitative evaluation of evolution of a learning analytics tool. Computers & Education 58, 1, 470--489.
[2]
Buckingham Shum, S. and Deakin Crick, R., 2012. Learning dispositions and transferable competencies: pedagogy, modelling and learning analytics. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge ACM, 92--101.
[3]
Chiu, M. M., 2008. Effects of argumentation on group micro-creativity: Statistical discourse analyses of algebra students' collaborative problem solving. Contemporary Educational Psychology 33, 3, 382--402.
[4]
Crowston, K., Allen, E. E., and Heckman, R., 2011. Using natural language processing technology for qualitative data analysis. International Journal of Social Research Methodology 15, 6 (2012/11/01), 523--543. DOI= http://dx.doi.org/10.1080/13645579.2011.625764.
[5]
Dawson, S. and Siemens, G., 2014. Analytics to literacies: The development of a learning analytics framework for multiliteracies assessment. In The International Review of Research in Open and Distributed Learning, 285--305.
[6]
Drach-Zahavy, A. and Somech, A., 2002. Team heterogeneity and its relationship with team support and team effectiveness. Journal of Educational Administration 40, 1, 44--66.
[7]
Duffy, T. and Jonassen, D., 1992. Constructivism and the technology of instruction: a conversation. Lawrence Erlbaum Associates Publishers, Hillsdale, N.J.
[8]
Duval, E., 2011. Attention please!: learning analytics for visualization and recommendation. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge ACM, 9--17.
[9]
Erkens, G. and Janssen, J., 2008. Automatic coding of dialogue acts in collaboration protocols. International Journal of Computer-Supported Collaborative Learning 3, 4, 447--470.
[10]
Ferguson, R., 2012. Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning 4, 5-6, 304--317.
[11]
Ferguson, R. and Buckingham Shum, S., 2012. Social learning analytics: five approaches. In Proceedings of the 2nd international conference on learning analytics and knowledge ACM, 23--33.
[12]
Freeman, M. and Mckenzie, J., 2002. SPARK, a confidential web--based template for self and peer assessment of student teamwork: benefits of evaluating across different subjects. British Journal of Educational Technology 33, 5, 551--569.
[13]
Garrison, D. R., Anderson, T., and Archer, W., 2001. Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of distance education 15, 1, 7--23.
[14]
Greller, W. and Drachsler, H., 2012. Translating learning into numbers: A generic framework for learning analytics. Journal of Educational Technology & Society 15, 3, 42--57.
[15]
Hung, D. and Nichani, M., 2001. Constructivism and e-learning: Balancing between the individual and social levels of cognition. Educational Technology 41, 2, 40--44.
[16]
Janssen, J. and Bodemer, D., 2013. Coordinated Computer-Supported Collaborative Learning: Awareness and Awareness Tools. Educational psychologist 48, 1 (2013/01/01), 40--55. DOI= http://dx.doi.org/10.1080/00461520.2012.749153.
[17]
Janssen, J., Erkens, G., and Kirschner, P. A., 2011. Group awareness tools: It's what you do with it that matters. Computers in Human Behavior 27, 3, 1046--1058.
[18]
Järvelä, S., Kirschner, P. A., Panadero, E., Malmberg, J., Phielix, C., Jaspers, J., Koivuniemi, M., and Järvenoja, H., 2015. Enhancing socially shared regulation in collaborative learning groups: designing for CSCL regulation tools. Educational Technology Research and Development 63, 1 (2015/02/01), 125--142. DOI= http://dx.doi.org/10.1007/s11423-014-9358-1.
[19]
Kayes, A. B., Kayes, D. C., and Kolb, D. A., 2005. Experiential learning in teams. Simulation & Gaming 36, 3 (September 1, 2005), 330--354. DOI= http://dx.doi.org/10.1177/1046878105279012.
[20]
Kirschner, P. A., Strijbos, J.-W., Kreijns, K., and Beers, P. J., 2004. Designing electronic collaborative learning environments. Educational Technology Research and Development 52, 3, 47--66.
[21]
Knight, S., Shum, S. B., and Littleton, K., 2014. Epistemology, assessment, pedagogy: where learning meets analytics in the middle space. Journal of Learning Analytics 1, 2, 23--47.
[22]
Koh, E., Hong, H., and Seah, J., 2014. An Analytic Frame and Multi-method Approach to Measure Teamwork Competency. In 14th International Conference on Advanced Learning Technologies (ICALT) IEEE, Athens, 264--266. DOI= http://dx.doi.org/
[23]
Kolb, D. A., 1984. Experiential learning: Experience as the source of learning and development. Prentice-Hall Englewood Cliffs, NJ.
[24]
Lockyer, L., Heathcote, E., and Dawson, S., 2013. Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist 57, 10, 1439--1459.
[25]
Loughry, M. L., Ohland, M. W., and Moore, D. D., 2007. Development of a Theory-Based Assessment of Team Member Effectiveness. Educational and Psychological Measurement 67, 3, 505--524. DOI= http://dx.doi.org/10.1177/0013164406292085.
[26]
Mayne, L., 2012. Reflective writing as a tool for assessing teamwork in bioscience: Insights into student performance and understanding of teamwork. Biochemistry and Molecular Biology Education 40, 4, 234--240.
[27]
Meier, A., Spada, H., and Rummel, N., 2007. A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning 2, 1, 63--86.
[28]
Phielix, C., Prins, F. J., and Kirschner, P. A., 2010. Awareness of group performance in a CSCL-environment: Effects of peer feedback and reflection. Computers in Human Behavior 26, 2, 151--161. DOI= http://dx.doi.org/10.1016/j.chb.2009.10.011.
[29]
Phielix, C., Prins, F. J., Kirschner, P. A., Erkens, G., and Jaspers, J., 2011. Group awareness of social and cognitive performance in a CSCL environment: Effects of a peer feedback and reflection tool. Computers in Human Behavior 27, 3, 1087--1102.
[30]
Prins, F. J., Sluijsmans, D. M., Kirschner, P. A., and Strijbos, J.-W., 2005. Formative peer assessment in a CSCL environment: A case study. Assessment & Evaluation in Higher Education 30, 4, 417--444.
[31]
Rodríguez-Triana, M. J., Martínez-Monés, A., Asensio-Pérez, J. I., and Dimitriadis, Y., 2015. Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology 46, 2, 330--343.
[32]
Rosé, C. P., Wang, Y.-C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., and Fischer, F., 2008. Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning 3, 3 (2008/09/01), 237--271. DOI= http://dx.doi.org/10.1007/s11412-007-9034-0.
[33]
Rummel, N., Spada, H., and Hauser, S., 2009. Learning to collaborate while being scripted or by observing a model. International Journal of Computer-Supported Collaborative Learning 4, 1, 69--92.
[34]
Salas, E., Rosen, M. A., Burke, C. S., and Goodwin, G. F., 2009. The wisdom of collectives in organizations: An update of the teamwork competencies. In Team Effectiveness in Complex Organizations. Cross-Disciplinary Perspectives and Approaches, E. SALAS, G. F. GOODWIN and C. S. BURKE Eds. Routledge/Taylor & Francis Group, New York, 39--79.
[35]
Salas, E., Sims, D. E., and Burke, C. S., 2005. Is there a "Big Five" in teamwork? Small Group Research 36, 5, 555--599.
[36]
Strijbos, J.-W., Martens, R. L., and Jochems, W. M. G., 2004. Designing for interaction: Six steps to designing computer-supported group-based learning. Computers & Education 42, 4, 403--424. DOI= http://dx.doi.org/http://dx.doi.org/10.1016/j.compedu.2003.10.004.
[37]
Valentine, M. A., Nembhard, I. M., and Edmondson, A. C., 2012. Measuring teamwork in health care settings: A review of survey instruments. Harvard Business School.
[38]
Van Den Bossche, P., Gijselaers, W. H., Segers, M., and Kirschner, P. A., 2006. Social and Cognitive Factors Driving Teamwork in Collaborative Learning Environments: Team Learning Beliefs and Behaviors. Small Group Research 37, 5 (October 1, 2006), 490--521. DOI= http://dx.doi.org/10.1177/1046496406292938.
[39]
Verbert, K., Duval, E., Klerkx, J., Govaerts, S., and Santos, J. L., 2013. Learning analytics dashboard applications. American Behavioral Scientist 57, 10, 1500--1509.
[40]
Voogt, J., Erstad, O., Dede, C., and Mishra, P., 2013. Challenges to learning and schooling in the digital networked world of the 21st century. Journal of Computer Assisted Learning 29, 5, 403--413.
[41]
Vygotsky, L. S., 1978. Mind and society: The development of higher mental processes Cambridge, MA: Harvard University Press.
[42]
Whitelock, D., Twiner, A., Richardson, J. T., Field, D., and Pulman, S., 2015. OpenEssayist: a supply and demand learning analytics tool for drafting academic essays. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge ACM, 208--212.
[43]
Willey, K. and Freeman, M., 2006. Improving teamwork and engagement: the case for self and peer assessment. Australasian Journal of Engineering Education 12, 2--19.
[44]
Wise, A. F., 2014. Designing pedagogical interventions to support student use of learning analytics. In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge ACM, 203--211.
[45]
Yukawa, J., 2006. Co-reflection in online learning: Collaborative critical thinking as narrative. International Journal of Computer-Supported Collaborative Learning 1, 2, 203--228.

Cited By

View all
  • (2024)University Students’ Perceptions of Peer Assessment in Oral PresentationsEducation Sciences10.3390/educsci1403022114:3(221)Online publication date: 22-Feb-2024
  • (2024)Everyone Matters: A Multimodal Learning Analysis Framework Based on Individual Time SeriesTechnology, Knowledge and Learning10.1007/s10758-024-09742-5Online publication date: 13-May-2024
  • (2023)Comparative Analysis of Various EDM/LA Techniques Used For Improving Quality in Higher Education System2023 10th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON)10.1109/UPCON59197.2023.10434831(82-88)Online publication date: 1-Dec-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
LAK '16: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge
April 2016
567 pages
ISBN:9781450341905
DOI:10.1145/2883851
Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. assessment
  2. collaboration
  3. dispositional analytics
  4. evaluation
  5. learning design
  6. pedagogical model
  7. teamwork
  8. teamwork competency
  9. twenty-first century skills

Qualifiers

  • Research-article

Funding Sources

Conference

LAK '16

Acceptance Rates

LAK '16 Paper Acceptance Rate 36 of 116 submissions, 31%;
Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)37
  • Downloads (Last 6 weeks)2
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)University Students’ Perceptions of Peer Assessment in Oral PresentationsEducation Sciences10.3390/educsci1403022114:3(221)Online publication date: 22-Feb-2024
  • (2024)Everyone Matters: A Multimodal Learning Analysis Framework Based on Individual Time SeriesTechnology, Knowledge and Learning10.1007/s10758-024-09742-5Online publication date: 13-May-2024
  • (2023)Comparative Analysis of Various EDM/LA Techniques Used For Improving Quality in Higher Education System2023 10th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON)10.1109/UPCON59197.2023.10434831(82-88)Online publication date: 1-Dec-2023
  • (2023)FORTAGONO: A Model for the Technological Mediation of the Teaching and Learning ProcessesIEEE Access10.1109/ACCESS.2023.325444111(64294-64323)Online publication date: 2023
  • (2023)Learning analytics for online game-Based learning: a systematic literature reviewBehaviour & Information Technology10.1080/0144929X.2023.2255301(1-28)Online publication date: 19-Sep-2023
  • (2023)Learning Analytics for Learning: Emerging International Trends and Case Studies from the Asia-PacificInternational Handbook on Education Development in the Asia-Pacific10.1007/978-981-19-6887-7_54(1367-1393)Online publication date: 16-Nov-2023
  • (2023)Learning Analytics for Learning: Emerging International Trends and Case Studies from the Asia-PacificInternational Handbook on Education Development in Asia-Pacific10.1007/978-981-16-2327-1_54-1(1-27)Online publication date: 23-May-2023
  • (2022)Towards a Pragmatic and Theory-Driven Framework for Multimodal Collaboration FeedbackLAK22: 12th International Learning Analytics and Knowledge Conference10.1145/3506860.3506898(507-513)Online publication date: 21-Mar-2022
  • (2021)Current State of Learning AnalyticsAdvancing the Power of Learning Analytics and Big Data in Education10.4018/978-1-7998-7103-3.ch001(1-28)Online publication date: 2021
  • (2021)Pedagogical Design: Bridging Learning Theory and Learning AnalyticsConception pédagogique : Rapprocher la théorie de l'apprentissage et l'analyse de l'apprentissageCanadian Journal of Learning and Technology10.21432/cjlt2795947:1Online publication date: 9-Aug-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media