skip to main content
10.1145/3544549.3583830acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
extended-abstract

Spelland: Situated Language Learning with a Mixed-Reality Spelling Game through Everyday Objects

Published: 19 April 2023 Publication History

Abstract

This work explores the use of mixed-reality (MR) technology to enable situated language learning using everyday objects in the environment around the learners. The learning method is based on Presentation, Practice, and Production (PPP), which cultivates the habit of independent learning through repetition, practice, and demonstration. In our game design, the learners first interact with real-world objects via MR, and the objects’ spelling and their pronunciation will appear (Presentation), the learners repeat the pronunciation (Practice) to collect the letters of this objects, and finally the learner use the collected letters to spell out the target words (Production), which then transform into interactive 3D objects. We designed the learning experience and content tools using gestural UI, voice input, and object-to-word engine. Children in the preliminary user study found the game to be immersive, helpful in learning the spelling of the everyday objects and the target words, and additionally showed increased interests in learning about other nearby objects after playing the game.

Supplementary Material

ZIP File (3583830.supp.zip)
VTT File (3544549.3583830-preview.vtt)
VTT File (3544549.3583830-video.vtt)
MP4 File (3544549.3583830-preview.mp4)
Video Preview
MP4 File (3544549.3583830-video.mp4)
Video Presentation of the Game

References

[1]
Darren Edge, Elly Searle, Kevin Chiu, Jing Zhao, and James A. Landay. 2011. MicroMandarin: mobile language learning in context. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). Association for Computing Machinery, New York, NY, USA, 3169–3178. https://doi.org/10.1145/1978942.1979413
[2]
Christian David Vazquez, Afika Ayanda Nyati, Alexander Luh, Megan Fu, Takako Aikawa, and Pattie Maes. 2017. Serendipitous Language Learning in Mixed Reality. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). Association for Computing Machinery, New York, NY, USA, 2172–2179. https://doi.org/10.1145/3027063.3053098
[3]
Fiona Draxler, Audrey Labrie, Albrecht Schmidt, and Lewis L. Chuang. 2020. Augmented Reality to Enable Users in Learning Case Grammar from Their Real-World Interactions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376537
[4]
M. Weerasinghe, "VocabulARy: Learning Vocabulary in AR Supported by Keyword Visualisations," in IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 11, pp. 3748-3758, Nov. 2022.
[5]
Ari Hautasaari, Takeo Hamada, Kuntaro Ishiyama, and Shogo Fukushima. 2020. VocaBura: A Method for Supporting Second Language Vocabulary Learning While Walking. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 4, Article 135 (December 2019), 23 pages. https://doi.org/10.1145/3369824
[6]
A. Ibrahim, B. Huynh, J. Downey, T. Höllerer, D. Chun and J. O'donovan, "ARbis Pictus: A Study of Vocabulary Learning with Augmented Reality," in IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 11, pp. 2867-2874, Nov. 2018.
[7]
J. Barreira, M. Bessa, L. C. Pereira, T. Adão, E. Peres and L. Magalhães, "MOW: Augmented Reality game to learn words in different languages: Case study: Learning English names of animals in elementary school," 7th Iberian Conference on Information Systems and Technologies (CISTI 2012), Madrid, Spain, 2012, pp. 1-6.
[8]
Santos, M.E.C., Lübke, A.i., Taketomi, T. Augmented reality as multimedia: the case for situated vocabulary learning. RPTEL 11, 4 (2016). https://doi.org/10.1186/s41039-016-0028-2
[9]
C. S. C. Dalim, T. Piumsomboon, A. Dey, M. Billinghurst and S. Sunar, "TeachAR: An Interactive Augmented Reality Tool for Teaching Basic English to Non-native Children," 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), Merida, Mexico, 2016, pp. 344-345.
[10]
Nunan, David. Research Methods in Language Learning. 1st ed. Cambridge University Press, 1992.
[11]
Thornbury, Scott. How to Teach Speaking. 1st ed. Pearson Education ESL, 2005.
[12]
Anuj Kumar, Pooja Reddy, Anuj Tewari, Rajat Agrawal, and Matthew Kam. 2012. Improving literacy in developing countries using speech recognition-supported games on mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). Association for Computing Machinery, New York, NY, USA, 1149–1158. https://doi.org/10.1145/2207676.2208564
[13]
Pintrich, P. R. (2003). A Motivational Science Perspective on the Role of Student Motivation in Learning and Teaching Contexts. Journal of Educational Psychology, 95(4), 667–686. https://doi.org/10.1037/0022-0663.95.4.667
[14]
Nan Chao, Shengge Yang, Yuxian Qin, Zeming Song, Zhaofan Su, and Xiaomei Nie. 2022. AR-Poetry: Enhancing Children's Motivation in Learning Classical Chinese Poetry via Interactive Augmented Reality. In The Ninth International Symposium of Chinese CHI (Chinese CHI 2021). Association for Computing Machinery, New York, NY, USA, 162–166. https://doi.org/10.1145/3490355.3490518
[15]
Lave, Jean, and Etienne Wenger. Situated Learning: Legitimate Peripheral Participation. Cambridge University Press, 1991.
[16]
Willis J. A framework for task-based learning. Longman; 1996.
[17]
Putri Yaniafari, R., Olivia, V. and ., S. 2022. The Potential of ASR for Improving English Pronunciation: A Review. KnE Social Sciences. 7, 7 (Mar. 2022), 281–289.

Cited By

View all
  • (2024)Exploring the Potential of Mixed Reality in Enhancing Language Acquisition for EFL ChildrenCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678388(776-780)Online publication date: 5-Oct-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems
April 2023
3914 pages
ISBN:9781450394222
DOI:10.1145/3544549
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Check for updates

Author Tags

  1. Language Learning
  2. Learning Experience
  3. Mixed Reality
  4. Situated Learning
  5. Tangible Interaction

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Conference

CHI '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)369
  • Downloads (Last 6 weeks)68
Reflects downloads up to 12 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Exploring the Potential of Mixed Reality in Enhancing Language Acquisition for EFL ChildrenCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678388(776-780)Online publication date: 5-Oct-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media