Skip to main content
SpringerLink
Log in
Book cover

Machine Learning Paradigms pp 131–156Cite as

Multimodal Learning Analytics in a Laboratory Classroom

  • Chapter
  • First Online:

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 158))

Abstract

Sophisticated research approaches and tools can help researchers to investigate the complex processes involved in learning in various settings. The use of video technology to record classroom practices, in particular, can be a powerful way for capturing and studying learning and related phenomena within a social setting such as the classroom. This chapter outlines several multimodal techniques to analyze the learning activities in a laboratory classroom. The video and audio recordings were processed automatically to obtain information rather than requiring manual coding. Moreover, these automated techniques are able to extract information with an efficiency that is beyond the capabilities of human-coders, providing the means to deal analytically with the multiple modalities that characterize the classroom. Once generated, the information provided by the different modalities is used to explain and predict high-level constructs such as students’ attention and engagement. This chapter not only presents the results of the analysis, but also describes the setting, hardware and software needed to replicate this analytical approach.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    HyperFace: Face landmarks and pose detection. https://github.com/takiyu/hyperface.

  2. 2.

    DockerFace: Face detection https://github.com/natanielruiz/dockerface.

References

  1. Amidon, E., Flanders, N.A.: The effects of direct and indirect teacher influence on dependent-prone students learning geometry. J. Educ. Psychol. 52(6), 286–291 (1961). https://doi.org/10.1037/h0043674

    Article  Google Scholar 

  2. Antonakis, J., Bendahan, S., Jacquart, P., Lalive, R.: On making causal claims: a review and recommendations. Leadersh. Q. 21(6), 1086–1120 (2010). https://doi.org/10.1016/j.leaqua.2010.10.010

    Article  Google Scholar 

  3. Beeby, T., Burkhardt, H., Fraser, R.: Systematic Classroom Analysis Notation. Shell Centre for Mathematics Education, Nottingham, England (1979)

    Google Scholar 

  4. Bezdek, J.C., Ehrlich, R., Full, W.: FCM: the fuzzy c-means clustering algorithm. Comput. Geosci. 10(2), 191–203 (1984). https://doi.org/10.1016/0098-3004(84)90020-7

    Article  Google Scholar 

  5. Blikstein, P.: Multimodal learning analytics. In: Suthers, D., Verbert, K., Duval, E., Ochoa, X. (eds.) Proceedings of the Third International Conference on Learning Analytics and Knowledge, Leuven, Belgium, pp. 102–106 (2013)

    Google Scholar 

  6. Bourke, S.F.: The teaching and learning of mathematics: national report of the second phase of the IEA Classroom Environment Study (ACER research monograph No. 25). Australian Council for Educational Research, Hawthorn, Victoria (1984)

    Google Scholar 

  7. Campbell, D.T.: Assessing the impact of planned social change (Occasional Paper Series No. 8). Western Michigan University, College of Education, Evaluation Center, Kalamazoo, MI (1976). https://eric.ed.gov/?id=ED303512

  8. Chan, M.C.E., Baker, A., Slee, R., Williamson, J.: Educational engagement through the middle years of schooling: Report for the In2COmmuinity Applied Learning Project. Victoria Institute, Melbourne, Australia (2015). https://www.vu.edu.au/sites/default/files/victoria-institute/pdfs/Educational%20Engagement%20through%20the%20Middle%20Years%20of%20Schooling%20%28web-version2%29.pdf

  9. Chan, M.C.E., Clarke, D.J.: Video-based research in a laboratory classroom. In: Xu, L., Aranda, G., Clarke, D. (eds.) Video-Based Research in Education: Cross-Disciplinary Perspectives. Routledge, New York (in press)

    Google Scholar 

  10. Chan, M.C.E., Clarke, D.J.: Learning research in a laboratory classroom: Complementarity and commensurability in juxtaposing multiple interpretive accounts. In: Dooley, T., Gueudet, G. (eds.) Proceedings of the Congress of European Research in Mathematics Education, Dublin, Ireland, pp. 2713–2720 (2017)

    Google Scholar 

  11. Chan, M.C.E., Clarke, D.J.: Structured affordances in the use of open-ended tasks to facilitate collaborative problem solving. ZDM Int. J. Math. Educ. 49, 951–963 (2017). https://doi.org/10.1007/s11858-017-0876-2

    Article  Google Scholar 

  12. Chan, M.C.E., Clarke, D.J., Cao, Y.: The social essentials of learning: an experimental investigation of collaborative problem solving and knowledge construction in mathematics classrooms in Australia and China. Math. Educ. Res. J. 30(1), 39–50 (2017). https://doi.org/10.1007/s13394-017-0209-3

    Article  Google Scholar 

  13. Clarke,D.J.: International comparative studies in mathematics education. Chapter 5 In: Bishop, A.J.,Clements, M.A.,Keitel, C.,Kilpatrick, J., Leung, F.K.S. (eds.) Second InternationalHandbook of Mathematics Education, pp. 145–186. Dordrecht, The Netherlands: Kluwer Academic Publishers (2003)

    Google Scholar 

  14. Clarke, D.J.: Studying the classroom negotiation of meaning: Complementary accounts methodology, Chapter 7. In: Teppo, A. (ed.) Qualitative Research Methods in Mathematics Education. Monograph Number 9 of the Journal for Research in Mathematics Education, pp. 98–111. NCTM, Reston, VA (1998)

    Google Scholar 

  15. Clarke, D.J. (ed.): Perspectives on Practice and Meaning in Mathematics and Science Classrooms. Kluwer Academic Publishers, Dordrecht, The Netherlands (2001)

    Google Scholar 

  16. Clarke, D.J.: Using cross-cultural comparison to interrogate the logic of classroom research in mathematics education. In: Kaur, B., Ho, W.K., Toh, T.L., Choy, B.H. (eds.) Proceedings of the 41st Conference of the International Group for the Psychology of Mathematics Education, vol. 1, pp. 13–28. PME, Singapore (2017)

    Google Scholar 

  17. Clarke, D.J., Keitel, C., Shimizu, Y. (eds.): Mathematics Classrooms in Twelve Countries: The Insider’s Perspective. Sense Publishers, Rotterdam, The Netherlands (2006)

    Google Scholar 

  18. Clarke, D.J., Mitchell, C., Bowman, P.: Optimising the use of available technology to support international collaborative research in mathematics classrooms. In: Janik, T., Seidel, T. (eds.) The Power of Video Studies in Investigating Teaching and Learning in the Classroom, pp. 39–60. Waxmann, New York (2009)

    Google Scholar 

  19. Clarke, D.J., Xu, L.H., Wan, M.E.V.: Spoken mathematics as an instructional strategy: The public discourse of mathematics classrooms in different countries. In: Kaur, B., Anthony, G., Ohtani, M., Clarke, D. (eds.) Student Voice in Mathematics Classrooms Around the World, pp. 13–31. Sense Publishers, Rotterdam, The Netherlands (2013)

    Chapter  Google Scholar 

  20. Cobb, P., Bauersfeld, H.: The Emergence of Mathematical Meaning: Interaction in Classroom Cultures. L. Erlbaum Associates, Hillsdale, NJ (1995)

    Google Scholar 

  21. Dringus, L.P.: Learning analytics considered harmful. J. Asynchronous Learn. Netw. 16(3), 87–100 (2012)

    Google Scholar 

  22. Erlwanger, S.H.: Case studies of children’s conceptions of mathematics. J. Child. Math. Behav. 1(3), 157–283 (1975)

    Google Scholar 

  23. Ferguson, R.: Learning analytics: drivers, developments and challenges. Int. J. Technol. Enhanc. Learn. 4(5–6), 304–317 (2012)

    Article  Google Scholar 

  24. Ferreiro, E., Teberosky, A.: Literacy before schooling (K. Goodman Castro Trans.) [Los sistemas de escritura en el desarrollo del niño]. Heinemann Educational Books, Exeter, NH (Original work published 1979) (1982)

    Google Scholar 

  25. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74(1), 59–109 (2004)

    Article  Google Scholar 

  26. Frydenberg, E., Ainley, M., Russell, V.J.: Student motivation and engagement. Sch. Issues Dig. 2 (2005)

    Google Scholar 

  27. Gibbs, R., Poskitt, J.: Student Engagement in the Middle Years of Schooling (Years 7–10): A Literature Review. New Zealand Ministry of Education, Wellington (2010). http://www.educationcounts.govt.nz/publications/schooling/student-engagement-in-the-middle-years-of-schooling-years-7-10-a-literature-review/introduction

  28. Good, T.L., Grouws, D.A.: Teaching effects: a process-product study in fourth-grade mathematics classrooms. J. Teach. Educ. 28(3), 49–54 (1977). https://doi.org/10.1177/002248717702800310

    Article  Google Scholar 

  29. Hart, L.E.: Classroom processes, sex of student, and confidence in learning mathematics. J. Res. Math. Educ. 20(3), 242–260 (1989). https://doi.org/10.2307/749514

    Article  Google Scholar 

  30. Helme, S., Clarke, D.: Cognitive engagement in the mathematics classroom. In: Clarke, D. (ed.) Perspectives on Practice and Meaning in Mathematics and Science Classrooms, pp. 131–153. Kluwer Academic Publishers, Dordrecht, The Netherlands (2001)

    Google Scholar 

  31. Hiebert, J., Gallimore, R., Garnier, H., Givvin, K.B., Hollingsworth, H., Jacobs, J., et al.: Teaching Mathematics in Seven Countries: Results from the TIMSS 1999 Video Study (NCES 2003–013 Revised). U.S. Department of Education, National Center for Education Statistics, Washington, DC (2003). http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2003013

  32. Janík, T., Seidel, T. (eds.): The Power of Video Studies in Investigating Teaching and Learning in the Classroom. Waxmann, Münster, Germany (2009)

    MATH  Google Scholar 

  33. Kleinsmith, A., Bianchi-Berthouze, N.: Recognizing affective dimensions from body posture. In: Paiva, A., Prada, R., Picard, R.W. (eds.) Proceedings of the Affective Computing and Intelligent Interaction Conference, Lisbon, Portugal, pp. 48–58 (2007)

    Google Scholar 

  34. Lim, F.V., O’Halloran, K.L., Podlasov, A.: Spatial pedagogy: mapping meanings in the use of classroom space. Camb. J. Educ. 42(2), 235–251 (2012)

    Article  Google Scholar 

  35. Marshall, S.: Exploring the ethical implications of MOOCs. Distance Educ. 35(2), 250–262 (2014). https://doi.org/10.1080/01587919.2014.917706

    Article  Google Scholar 

  36. Mota, S., Picard, R.W.: Automated posture analysis for detecting learner’s interest level. In: Martinez, A.M., Tan, H.Z. (eds.) Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop, Madison, WI, vol. 5, pp. 49–54, June 2003

    Google Scholar 

  37. Ochoa, X., Chiluiza, K., Méndez, G., Luzardo, G., Guamán, B., Castells, J.: Expertise estimation based on simple multimodal features. In: Proceedings of the 15th ACM on International Conference on Multimodal Interaction, pp. 583–590. ACM, Dec 2013

    Google Scholar 

  38. Ochoa, X.: Multimodal learning analytics. In: Lang, C., Siemens, G., Wise, A.F., Gaševic, D. (eds.) The Handbook of Learning Analytics, pp. 129–141. Society for Learning Analytics Research (SoLAR), Alberta, Canada (2017)

    Chapter  Google Scholar 

  39. Ochoa, X., Worsley, M.: Augmenting learning analytics with multimodal sensory data. J. Learn. Anal. 3(2), 213–219 (2016)

    Article  Google Scholar 

  40. Overholt, G.: Ethnography and education: limitations and sources of error. J. Thought 15(3), 11–20 (1980)

    Google Scholar 

  41. Paiva, A., Prada, R., Picard, R.W. (eds.): Conference Proceedings of the Second Affective Computing and Intelligent Interaction Conference. Springer, Lisbon, Portugal (2007)

    Google Scholar 

  42. Piaget, J.: The Language and Thought of the Child. Routledge & Kegan Paul, London (1926)

    Google Scholar 

  43. Peterson, P.L., Fennema, E.: Effective teaching, student engagement in classroom activities, and sex-related differences in learning mathematics. Am. Educ. Res. J. 22(3), 309–335 (1985). https://doi.org/10.2307/1162966

    Article  Google Scholar 

  44. Raca, M., Dillenbourg, P.: System for assessing classroom attention. In: Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp. 265–269. ACM, Apr 2013

    Google Scholar 

  45. Raca, M., Dillenbourg, P.: System for assessing classroom attention. In: Suthers, D., Verbert, K., Duval, E., Ochoa, X. (eds.) Proceedings of the Third International Conference on Learning Analytics and Knowledge, Leuven, Belgium, pp. 265–269, Apr 2013

    Google Scholar 

  46. Raca, M., Tormey, R., Dillenbourg, P.: Sleepers’ lag-study on motion and attention. In: Proceedings of the Fourth International Conference on Learning Analytics and Knowledge, Indianapolis, IN, pp. 36–43, Mar 2014

    Google Scholar 

  47. Sfard, A., Kieran, C.: Cognition as communication: Rethinking learning-by-talking through multi-faceted analysis of students’ mathematical interactions. Mind Cult. Act. 8(1), 42–76 (2001). https://doi.org/10.1207/s15327884mca0801_04

    Article  Google Scholar 

  48. Shernoff, D.J.: Optimal Learning Environments to Promote Student Engagement. Springer, New York (2013)

    Book  Google Scholar 

  49. Skinner, B.F.: The science of learning and the art of teaching. Harv. Educ. Rev. 24, 86–97 (1954)

    Google Scholar 

  50. Skinner, E.A., Belmont, M.J.: Motivation in the classroom: reciprocal effects of teacher behavior and student engagement across the school year. J. Educ. Psychol. 85(4), 571–581 (1993)

    Article  Google Scholar 

  51. Stigler, J.W., Hiebert, J.: The Teaching Gap: Best Ideas from the World’s Teachers for Improving Education in the Classroom. Free Press, New York (2009)

    Google Scholar 

  52. Taylor, L., Parsons, J.: Improving student engagement. Curr. Issues Educ. 14(1) (2011)

    Google Scholar 

  53. Ulewicz, M., Beatty, A.: The Power of Video Technology in International Comparative Research in Education. National Academy Press, Washington, DC (2001)

    Google Scholar 

  54. Vygotsky, L.S.: Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge, MA (1978)

    Google Scholar 

Download references

Acknowledgements

This research was conducted with Science of Learning Research Centre funding provided by the Australian Research Council Special Initiatives Grant (SR120300015) and the Discovery Projects funding scheme (DP170102541). We would like to thank the students, parents, teachers, and school staff for their invaluable support of this project. We are also very grateful to our technical team, Cameron Mitchell and Peter (Reggie) Bowman, for their expertise in operating the Science of Learning Research Classroom facility for the project.

Author information

Authors and Affiliations

  1. Melbourne Graduate School of Education, The University of Melbourne, Victoria, Australia

    Man Ching Esther Chan & David Clarke

  2. New York University, New York, USA

    Xavier Ochoa

Authors
  1. Man Ching Esther Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xavier Ochoa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Clarke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Man Ching Esther Chan .

Editor information

Editors and Affiliations

  1. Department of Informatics, University of Piraeus, Piraeus, Greece

    Maria Virvou

  2. Department of Informatics, University of Piraeus, Piraeus, Greece

    Efthimios Alepis

  3. University of Piraeus, Pireas, Greece

    George A. Tsihrintzis

  4. Faculty of Science, Technology and Mathematics, University of Canberra, Canberra, SA, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Chan, M.C.E., Ochoa, X., Clarke, D. (2020). Multimodal Learning Analytics in a Laboratory Classroom. In: Virvou, M., Alepis, E., Tsihrintzis, G., Jain, L. (eds) Machine Learning Paradigms. Intelligent Systems Reference Library, vol 158. Springer, Cham. https://doi.org/10.1007/978-3-030-13743-4_8

Download citation

Publish with us

Policies and ethics

Search

Navigation