Skip to main content

Mixed Reality Agents as Language Learning Tutors

  • Conference paper
  • First Online:
Augmented Intelligence and Intelligent Tutoring Systems (ITS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13891))

Included in the following conference series:

Abstract

A challenge of language learning is to support its content with visual elements to add visual memories to textual and auditory recall. Whereas the depiction of nouns that portray objects is straightforward, verbs which describe actions require actors to clearly communicate the content. Here, mixed reality agents can guide the learner by demonstrating the action. In this paper, we investigate how such mixed reality agents can be utilized for language learning by combining them with the language learning method of total physical response. We compare the resulting open-source application to traditional flashcards in a between-subjects user study. There is no significant difference between the learner’s performance with mixed reality and the flashcards but students preferred the mixed reality application with regard to engagement and technology acceptance. The study highlights the potential of accessible mixed reality agents which can be expanded into general tutoring systems with personalized tasks and individual feedback.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/rwth-acis/ar-tutor.

References

  1. Asher, J.J.: The total physical response approach to second language learning. Mod. Lang. J. 53(1), 3 (1969). https://doi.org/10.2307/322091

    Article  Google Scholar 

  2. Bangor, A., Kortum, P., Miller, J.: Determining what individual SUS scores mean: adding an adjective rating scale. J. Usability Stud. 4(3), 114–123 (2009). http://uxpajournal.org/determining-what-individual-sus-scores-mean-adding-an-adjective-rating-scale/

  3. Brooke, J.: SUS: A Quick and Dirty Usability Scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, I.L. (eds.) Usability Evaluation in Industry, pp. 189–194. Taylor & Francis (1996), https://www.taylorfrancis.com/chapters/edit/10.1201/9781498710411-35/sus-quick-dirty-usability-scale-john-brooke

  4. Hensen, B., Bekhter, D., Blehm, D., Meinberger, S., Klamma, R.: Mixed reality agents for automated mentoring processes. In: de Paolis, L.T., Arpaia, P., Sacco, M. (eds.) XR Salento 2022. LNCS, vol. 13446, pp. 3–16. Springer Nature Switzerland, Cham (2022). https://doi.org/10.1007/978-3-031-15553-6_1

    Chapter  Google Scholar 

  5. Holz, T., Campbell, A.G., O’Hare, G.M., Stafford, J.W., Martin, A., Dragone, M.: MiRA–mixed reality agents. Int. J. Hum.-Comput. Stud. 69(4), 251–268 (2011). https://doi.org/10.1016/j.ijhcs.2010.10.001

    Article  Google Scholar 

  6. Huynh, B., Orlosky, J., Hollerer, T.: In-situ labeling for augmented reality language learning. In: Teather, R., Itoh, Y., Gabbard, J. (eds.) Proceedings, 26th IEEE Conference on Virtual Reality and 3D User Interfaces, pp. 1606–1611. IEEE, Piscataway, NJ (2019). https://doi.org/10.1109/VR.2019.8798358

  7. Ibrahim, A., Huynh, B., Downey, J., Hollerer, T., Chun, D., O’Donovan, J.: ARbis pictus: a study of vocabulary learning with augmented reality. IEEE Trans. Visual Comput. Graphics 24(11), 2867–2874 (2018). https://doi.org/10.1109/TVCG.2018.2868568

    Article  Google Scholar 

  8. Jia, T., Liu, Y.: Words in kitchen: an instance of leveraging virtual reality technology to learn vocabulary. In: Adjunct proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, pp. 150–155. IEEE Computer Society, Conference Publishing Services, Los Alamitos, California (2019). https://doi.org/10.1109/ISMAR-Adjunct.2019.00-59

  9. Kallmann, M., Thalmann, D.: Modeling behaviors of interactive objects for real-time virtual environments. J. Vis. Lang. Comput. 13(2), 177–195 (2002). https://doi.org/10.1006/jvlc.2001.0229

    Article  Google Scholar 

  10. Martinez, A.A., Benito, J.R.L., Gonzalez, E.A., Ajuria, E.B.: An experience of the application of Augmented Reality to learn English in Infant Education. In: 2017 International Symposium on Computers in Education (SIIE), pp. 1–6. IEEE (2017). https://doi.org/10.1109/SIIE.2017.8259645

  11. Park, S.Y.: An analysis of the technology acceptance model in understanding university students’ behavioral intention to use e-learning. J. Educ. Technol. Soc. 12(3), 150–162 (2009). http://www.jstor.org/stable/jeductechsoci.12.3.150

  12. Roediger, H.L., Karpicke, J.D.: Test-enhanced learning: taking memory tests improves long-term retention. Psychol. Sci. 17(3), 249–255 (2006). https://doi.org/10.1111/j.1467-9280.2006.01693.x

    Article  Google Scholar 

Download references

Acknowledgements

We thank the German Federal Ministry of Education and Research for their support within the project “Personalisierte Kompetenzentwicklung und hybrides KI-Mentoring” (tech4compKI; id: 16DHB2213).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Benedikt Hensen .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hensen, B., Bekhter, D. (2023). Mixed Reality Agents as Language Learning Tutors. In: Frasson, C., Mylonas, P., Troussas, C. (eds) Augmented Intelligence and Intelligent Tutoring Systems. ITS 2023. Lecture Notes in Computer Science, vol 13891. Springer, Cham. https://doi.org/10.1007/978-3-031-32883-1_50

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-32883-1_50

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-32882-4

  • Online ISBN: 978-3-031-32883-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics