skip to main content
10.1145/3371382.3378350acmconferencesArticle/Chapter ViewAbstractPublication PageshriConference Proceedingsconference-collections
abstract

Effects of Pitch Gestures on Learning Chinese Orthography with a Social Robot

Published: 01 April 2020 Publication History

Abstract

In this study, we investigate the effect of a social robot using head and arm gestures mimicking the lexical tones (i.e. pitch gestures) on learning pronunciations and translations of Chinese characters. Performance was compared between two within-subjects conditions: Gesture Observation condition, in which the robot used pitch gestures to teach six characters and the No Gesture condition in which the robot did not use gestures to teach six characters. Participants (N = 21) were tested on how well they pronounced and translated the learned characters. The study showed that a robot not using gestures was found to enhance learning, but only when participants could first familiarize with learning Chinese characters with the robot using pitch gestures. These results suggested that prior knowledge of learning Chinese attained from a robot using pitch gestures improved recall on learning the characters during a learning module with a robot not using gestures.

References

[1]
Rianne van den Berghe, Josje Verhagen, Ora Oudgenoeg-Paz, Sanne van der Ven, and Paul Leseman. 2018. Social robots for language learning: A review. Review of Educational Research, 89, 259--295.
[2]
Jan de Wit, Thorsten Schodde, Bram Willemsen, Kirsten Bergmann, Mirjam de Haas, Stefan Kopp, Emiel Krahmer, and Paul Vogt. 2018. The effect of a robot's gestures and adaptive tutoring on children's acquisition of second language vocabularies. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (pp. 50--58). ACM.
[3]
Paul Vogt, Rianne van den Berghe, Mirjam de Haas, Laura Hoffmann, Junko Kanero, Ezgi Mamus, et al. 2019. "Second language tutoring using social robots. A large-scale study," in IEEE/ACM Int. Conf. on Human- Robot Interaction (HRI 2019) (Daegu). 73077
[4]
Yi Xu. 1994. Production and perception of coarticulated tones. Journal of the Acoustical Society of America, 95, 2240--2253.
[5]
Mirjam Broersma and Anne Cutler. 2011. Competition dynamics of second-language listening. Quarterly Journal of Experimental Psychology, 64, 74--95.
[6]
Pierre A. Hallé, Yueh-Chin Chang, and Catherine T. Best. 2004. Identification and discrimination of Mandarin Chinese tones by Mandarin Chinese vs. French listeners. Journal of Phonetics, 32, 395--421.
[7]
Laura M. Morett and Li-Yun Chang. 2015. Emphasising sound and meaning: pitch gestures enhance Mandarin lexical tone acquisition. Language, Cognition and Neuroscience, 30, 347--353.
[8]
Trevor H. Chen and Dominic W. Massaro. 2008. Seeing pitch: Visual information for lexical tones of Mandarin-Chinese. The Journal of the Acoustical Society of America, 123, 2356--2366.
[9]
Jingna Li and Yao Wang. 2013. A study of accentedness in the speech of Chinese EFL Learners. Canadian Social Science, 9, 150--155.
[10]
Telle Hailikari, Nina Katajavuori, and Sari Lindblom-Ylänne. 2008. The relevance of prior knowledge in learning and instructional design. American Journal of Pharmaceutical Education 72, 1--8.
[11]
Helen H. Shen. 2013. Chinese L2 literacy development: cognitive characteristics, learning strategies, and pedagogical interventions. Language and Linguistics Compass, 7, 371--387.

Cited By

View all
  • (2023)Illustrating Robot MovementsProceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3568162.3576956(231-242)Online publication date: 13-Mar-2023
  • (2023)Can a Robot Companion Help Students Learn Chinese Tones? The Role of Speech and Gesture CuesCreativity in Intelligent Technologies and Data Science10.1007/978-3-031-44615-3_29(420-432)Online publication date: 14-Oct-2023
  • (2022)The Design and Observed Effects of Robot-performed Manual Gestures: A Systematic ReviewACM Transactions on Human-Robot Interaction10.1145/354953012:1(1-62)Online publication date: 19-Jul-2022

Index Terms

  1. Effects of Pitch Gestures on Learning Chinese Orthography with a Social Robot

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    HRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction
    March 2020
    702 pages
    ISBN:9781450370578
    DOI:10.1145/3371382
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 April 2020

    Check for updates

    Author Tags

    1. chinese orthography
    2. gestures
    3. robot tutoring
    4. second language learning

    Qualifiers

    • Abstract

    Conference

    HRI '20
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 192 of 519 submissions, 37%

    Upcoming Conference

    HRI '25
    ACM/IEEE International Conference on Human-Robot Interaction
    March 4 - 6, 2025
    Melbourne , VIC , Australia

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)7
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Illustrating Robot MovementsProceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3568162.3576956(231-242)Online publication date: 13-Mar-2023
    • (2023)Can a Robot Companion Help Students Learn Chinese Tones? The Role of Speech and Gesture CuesCreativity in Intelligent Technologies and Data Science10.1007/978-3-031-44615-3_29(420-432)Online publication date: 14-Oct-2023
    • (2022)The Design and Observed Effects of Robot-performed Manual Gestures: A Systematic ReviewACM Transactions on Human-Robot Interaction10.1145/354953012:1(1-62)Online publication date: 19-Jul-2022

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media