Skip to main content
Springer Nature Link
Log in

How Peers Communicate Without Words-An Exploratory Study of Hand Movements in Collaborative Learning Using Computer-Vision-Based Body Recognition Techniques

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13916))

Included in the following conference series:

  • 4743 Accesses

Abstract

Accumulating research in embodied cognition highlights the essential role of human bodies in knowledge learning and development. Hand movement is one of the most applied body motions in the collaborative ideation task when students co-construct knowledge with and without words. However, there is a limited understanding of how students in a group use their hand movements to coordinate understandings and reach a consensus. This study explored students’ hand movement patterns during the different types of knowledge co-construction discourses: quick consensus-building, integration-oriented consensus building, and conflict-oriented consensus building. Students’ verbal discussion transcripts were qualitatively analyzed to identify the type of knowledge co-construction discourses. Students’ hand motion was video-recorded, and their hand landmarks were detected using the machine learning tool MediaPipe. One-way ANOVA was conducted to compare students hand motions in different types of discourses. The results found there were different hand motion patterns in different types of collaboration discourses. Students tended to employ more hand motion during conflict-oriented consensus building discourses than during quick consensus building and integration-oriented consensus building discourses. At the group level, the collaborating students were found to present less equal hand movement during quick consensus-building than integration-oriented consensus building and conflict-oriented consensus building. The findings expand the existing understanding of embodied collaborative learning, providing insights for optimizing collaborative learning activities incorporating both verbal and non-verbal language.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. OECD.: Reviews of National Policies for Education Education in Chile. OECD Publishing (2017)

    Google Scholar 

  2. Roschelle, J., Teasley, S.D.: The construction of shared knowledge in collaborative problem solving. In: Computer supported collaborative learning. Springer (1995)

    Google Scholar 

  3. Scardamalia, M., Bereiter, C.: Computer support for knowledge-building communities. J. Learn. Sci. 3(3), 265–283 (1994)

    Article  Google Scholar 

  4. Johnson, D.W., Johnson, F.P.: Joining Together: Group Theory and Group Skills. Prentice-Hall, Inc. (1991)

    Google Scholar 

  5. Hull, D.M., Saxon, T.F.: Negotiation of meaning and co-construction of knowledge: an experimental analysis of asynchronous online instruction. Comput. Educ. 52(3), 624–639 (2009)

    Article  Google Scholar 

  6. Stump, G.S., et al.: Collaborative learning in engineering students: gender and achievement. J. Eng. Educ. 100(3), 475–497 (2011)

    Article  Google Scholar 

  7. Marra, R.M., et al.: Beyond “group work”: an integrated approach to support collaboration in engineering education. Int. J. STEM Educ. 3(1), 1–15 (2016)

    Article  Google Scholar 

  8. Du, X., et al.: Engineering students’ conceptions of collaboration, group-based strategy use, and perceptions of assessment in PBL: a case study in Qatar. Int. J. Eng. Educ. 36(1), 296–308 (2020)

    Google Scholar 

  9. Nguyen, A., et al.: Exploring socially shared regulation with an AI deep learning approach using multimodal data. In: Proceedings of International Conferences of Learning Sciences (ICLS) (2022)

    Google Scholar 

  10. Zhu, G., et al.: Exploring emotional and cognitive dynamics of knowledge building in grades 1 and 2. User Model. User-Adap. Inter. 29(4), 789–820 (2019)

    Article  Google Scholar 

  11. Lakoff, G., Núñez, R.: Where Mathematics Comes from, vol. 6. Basic Books, New York (2000)

    MATH  Google Scholar 

  12. Nathan, M.J., et al.: Teachers’ attitudes about gesture for learning and instruction. Gesture 18(1), 31–56 (2019)

    Article  Google Scholar 

  13. Hidalgo, G., et al.: OpenPose library. CMU-Perceptual-Computing-Lab/https://github.com/CMU-Perceptual-Computing-Lab. (2017)

  14. Güler, R.A., Neverova, N., Kokkinos, I.: Densepose: dense human pose estimation in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition (2018)

    Google Scholar 

  15. Schneider, B., Worsley, M., Martinez-Maldonado, R.: Gesture and gaze: multimodal data in dyadic interactions. In: International Handbook of Computer-Supported Collaborative Learning, pp. 625–641. Springer (2021)

    Chapter  Google Scholar 

  16. Weinberger, A., Fischer, F.: A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Comput. Educ. 46(1), 71–95 (2006)

    Article  Google Scholar 

  17. De Wever, B., et al.: Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Comput. Educ. 46(1), 6–28 (2006)

    Article  Google Scholar 

  18. Strijbos, J.-W., et al.: Content analysis: what are they talking about? Comput. Educ. 46(1), 29–48 (2006)

    Article  Google Scholar 

  19. Jewitt, C.: Technology, Literacy, Learning: A Multimodal Approach. Routledge (2012)

    Google Scholar 

  20. Mangaroska, K., et al.: Challenges and opportunities of multimodal data in human learning: the computer science students’ perspective. J. Comput. Assist. Learn. 37(4), 1030–1047 (2021)

    Article  Google Scholar 

  21. Barsalou, L.W., et al.: Social embodiment. Psychol Learn. Motiv. 43, 43–92 (2003)

    Article  Google Scholar 

  22. Shapiro, L., Stolz, S.A.: Embodied cognition and its significance for education. Theory Res. Educ. 17(1), 19–39 (2019)

    Article  Google Scholar 

  23. Kiverstein, J.: The meaning of embodiment. Top. Cogn. Sci. 4(4), 740–758 (2012)

    Article  Google Scholar 

  24. Peng, Z., et al.: Recognizing teachers’ hand gestures for effective non-verbal interaction. Appl. Sci. 12(22), 11717 (2022)

    Article  Google Scholar 

  25. Schwartz, D.L., Black, J.B.: Shuttling between depictive models and abstract rules: induction and fallback. Cogn. Sci. 20(4), 457–497 (1996)

    Article  Google Scholar 

  26. Yammiyavar, P., Clemmensen, T., Kumar, J.: Analyzing non-verbal cues in usability evaluation tests. In: International Conference on Usability and Internationalization Springer (2007)

    Google Scholar 

  27. Goldin-Meadow, S., Singer, M.A.: From children’s hands to adults’ ears: gesture’s role in the learning process. Dev. Psychol. 39(3), 509 (2003)

    Article  Google Scholar 

  28. Cook, S.W., et al.: Hand gesture and mathematics learning: lessons from an Avatar. Cogn. Sci. 41(2), 518–535 (2017)

    Article  Google Scholar 

  29. Cash, P., Maier, A.: Prototyping with your hands: the many roles of gesture in the communication of design concepts. J. Eng. Des. 27(1–3), 118–145 (2016)

    Article  Google Scholar 

  30. Sakamoto, T., et al.: Radar-based hand gesture recognition using IQ echo plot and convolutional neural network. In: 2017 IEEE Conference on Antenna Measurements & Applications (CAMA), IEEE (2017)

    Google Scholar 

  31. Molchanov, P., et al.: Hand gesture recognition with 3D convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (2015)

    Google Scholar 

  32. Pisharady, P.K., Saerbeck, M.: Recent methods and databases in vision-based hand gesture recognition: a review. Comput. Vis. Image Underst. 141, 152–165 (2015)

    Article  Google Scholar 

  33. Zhang, F., et al.: Mediapipe hands: on-device real-time hand tracking. arXiv preprint arXiv:2006.10214 (2020)

  34. Nguyen, H.: Let’s teach Kibot: Discovering discussion patterns between student groups and two conversational agent designs. Brit. J. Educ. Technol. 53, 1864–1884 (2022)

    Article  Google Scholar 

  35. Tan, E., de Weerd, J.G., Stoyanov, S.: Supporting interdisciplinary collaborative concept mapping with individual preparation phase. Educ. Tech. Res. Dev. 69(2), 607–626 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Institute of Education, Nanyang Technological University, Singapore, 637616, Singapore

    Qianru Lyu, Wenli Chen, Junzhu Su & Kok Hui John Gerard Heng

  2. Tianjin University, Beiyang Avenue 300072, Tianjin, China

    Shuai Liu

Authors
  1. Qianru Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenli Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junzhu Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kok Hui John Gerard Heng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qianru Lyu .

Editor information

Editors and Affiliations

  1. University of Southern California, Los Angeles, CA, USA

    Ning Wang

  2. University of British Columbia, Vancouver, BC, Canada

    Genaro Rebolledo-Mendez

  3. North Carolina State University, Raleigh, NC, USA

    Noboru Matsuda

  4. Despacho 3.01, UNED-Grupo de Investigación aDeNu, Madrid, Spain

    Olga C. Santos

  5. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lyu, Q., Chen, W., Su, J., Heng, K.H.J.G., Liu, S. (2023). How Peers Communicate Without Words-An Exploratory Study of Hand Movements in Collaborative Learning Using Computer-Vision-Based Body Recognition Techniques. In: Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O.C., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2023. Lecture Notes in Computer Science(), vol 13916. Springer, Cham. https://doi.org/10.1007/978-3-031-36272-9_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-36272-9_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-36271-2

  • Online ISBN: 978-3-031-36272-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics