skip to main content
10.1145/3568294.3580176acmconferencesArticle/Chapter ViewAbstractPublication PageshriConference Proceedingsconference-collections
short-paper

Language Learning using Caption Generation within Reciprocal Multi-Party Child-Tutor-Tutee Interaction

Published:13 March 2023Publication History

ABSTRACT

Reciprocal Peer Tutoring (RPT) is a learning paradigm characteristic of collaborative interaction between learners with alternating tutor-tutee roles. In recent years, robot-assisted language learning (RALL) has gained traction by its wide application for learning language skills, such as speaking or writing, using social robots. Our work aims at exploring the effectiveness of RPT in learning Kazakh as a second language with the help of two robots acting either as a tutee or a tutor. To this end, we piloted a within-subject experiment with 21 children aged 8 and 9 years old from a primary school with Kazakh and Russian languages of instruction. The results show that the tutor robot was more effective in terms of learning gains, while the tutee robot brought positive emotional experiences.

Skip Supplemental Material Section

Supplemental Material

HRI23-fp1248.mp4

mp4

3.2 MB

References

  1. 2022. Tacotron 2 And WaveGlow v1.10 For PyTorch. https://github.com/NVIDIA/ DeepLearningExamples/tree/master/\PyTorch/SpeechSynthesis/Tacotron2Google ScholarGoogle Scholar
  2. Amy Roseman Allen and Nancy Boraks. 1978. Peer Tutoring: Putting It to the Test. The Reading Teacher, Vol. 32, 3 (1978), 274--278. http://www.jstor.org/stable/20194752Google ScholarGoogle Scholar
  3. Shuang Bai and Shan An. 2018. A survey on automatic image caption generation. Neurocomputing, Vol. 311 (2018), 291--304.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Karen Linn Bierman and Wyndol Furman. 1981. Effects of Role and Assignment Rationale on Attitudes Formed During Peer Tutoring. J Educ Psychol, Vol. 73, 33--40. https://doi.org/10.1037/0022-0663.73.1.33Google ScholarGoogle ScholarCross RefCross Ref
  5. Catherine C. Boudouris. 2005. Peer-Tutoring: Positive Peer Interactions. The Ohio Reading Teacher, Vol. 37 (2005), 11.Google ScholarGoogle Scholar
  6. Katja M. Mayer Leona Sureth Andrea Klingebiel Gesa Hartwigsen Manuela Macedonia Katharina von Kriegstein Brian Mathias, Christian Andrä. 2020. How Can We Learn Foreign Language Vocabulary More Easily? Frontiers for Young Minds, Vol. 8, 89. https://doi.org/10.3389/frym.2020.00089Google ScholarGoogle ScholarCross RefCross Ref
  7. Chih-Wei Chang, Jih-Hsien Lee, Po-Yao Chao, Chin-Yeh Wang, and Gwo-Dong Chen. 2010. Exploring the possibility of using humanoid robots as instructional tools for teaching a second language in primary school. Journal of Educational Technology & Society, Vol. 13, 2 (2010), 13--24.Google ScholarGoogle Scholar
  8. Huili Chen, Hae Won Park, and Cynthia Breazeal. 2020. Teaching and learning with children: Impact of reciprocal peer learning with a social robot on children's learning and emotive engagement. Computers & Education, Vol. 150 (2020). https://doi.org/10.1016/j.compedu.2020.103836Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Sungjin Lee, Hyungjong Noh, Jonghoon Lee, Kyusong Lee, Gary Lee, Seongdae Sagong, and Munsang Kim. 2011. On the effectiveness of Robot-Assisted Language Learning. ReCALL, Vol. 23 (2011), 25--58. https://doi.org/10.1017/S0958344010000273Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Panos Markopoulos, Janet C Read, Stuart MacFarlane, and Johanna Hoysniemi. 2008. Evaluating Children's Interactive Products: Principles and Practices for Interaction Designers. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.Google ScholarGoogle Scholar
  11. Javier R. Movellan, Micah Eckhardt, Marjo Virnes, and Angelica Rodriguez. 2009. Sociable robot improves toddler vocabulary skills. In 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI). 307--308. https://doi.org/10.1145/1514095.1514189Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Ryan Prenger, Rafael Valle, and Bryan Catanzaro. 2019. Waveglow: A Flow-based Generative Network for Speech Synthesis. 3617--3621. https://doi.org/10.1109/ICASSP.2019.8683143Google ScholarGoogle ScholarCross RefCross Ref
  13. Mitesh Puthran. 2018. Image Caption Generator. https://github.com/MiteshPuthran/Image-Caption-Generator/blob/master/Image20Captioning208k.ipynbGoogle ScholarGoogle Scholar
  14. Natasha Randall. 2019. A survey of robot-assisted language learning (RALL). ACM Transactions on Human-Robot Interaction (THRI), Vol. 9, 1 (2019), 1--36.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Grishma Sharma, Priyanka Kalena, Nishsi Malde, Aromal Nair, and Saurabh Parkar. 2019. Visual Image Caption Generator Using Deep Learning. 2nd International Conference on Advances in Science & Technology (2019).Google ScholarGoogle Scholar
  16. Jonathan Shen, Ruoming Pang, Ron J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, Rif A. Saurous, Yannis Agiomvrgiannakis, and Yonghui Wu. 2018. Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 4779--4783. https://doi.org/10.1109/ICASSP.2018.8461368Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Allen Thurston, David Duran, Erika Cunningham, Sílvia Blanch, and Keith Topping. 2009. International on-line reciprocal peer tutoring to promote modern language development in primary schools. Computers & Education, Vol. 53 (2009), 462--472. https://doi.org/10.1016/j.compedu.2009.03.005Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Keith Topping and Stewart Ehly. 2001. Peer Assisted Learning: A Framework for Consultation. Journal of Educational and Psychological Consultation - J EDUC PSYCHOLOGICAL CONS, Vol. 12 (06 2001), 113--132. https://doi.org/10.1207/S1532768XJEPC1202_03Google ScholarGoogle ScholarCross RefCross Ref
  19. Keith J. Topping and Angela Bryce. 2004. Cross? Age Peer Tutoring of Reading and Thinking: Influence on thinking skills. Educational Psychology, Vol. 24, 5 (2004), 595--621. https://doi.org/10.1080/0144341042000262935Google ScholarGoogle ScholarCross RefCross Ref
  20. Rianne van den Berghe. 2022. Social robots in a translanguaging pedagogy: A review to identify opportunities for robot-assisted (language) learning. Frontiers in Robotics and AI, Vol. 9 (2022). https://doi.org/10.3389/frobt.2022.958624Google ScholarGoogle ScholarCross RefCross Ref
  21. Rianne van den Berghe, Josje Verhagen, Ora Oudgenoeg-Paz, Sanne H. G. van der Ven, and Paul P. M. Leseman. 2018. Social Robots for Language Learning: A Review. Review of Educational Research, Vol. 89 (2018), 259--295.Google ScholarGoogle ScholarCross RefCross Ref
  22. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3156--3164.Google ScholarGoogle ScholarCross RefCross Ref
  23. Hsiu-Ping Yueh, Weijane Lin, S-Chen Wang, and Lily Fu. 2020. Reading with robot and human companions in library literacy activities: A comparison study. Br. J. Educ. Technol., Vol. 51 (2020), 1884--1900.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Language Learning using Caption Generation within Reciprocal Multi-Party Child-Tutor-Tutee Interaction

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction
          March 2023
          612 pages
          ISBN:9781450399708
          DOI:10.1145/3568294

          Copyright © 2023 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 13 March 2023

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • short-paper

          Acceptance Rates

          Overall Acceptance Rate242of1,000submissions,24%

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader