What matters in AI-supported learning: A study of human-AI interactions in language learning using cluster analysis and epistemic network analysis
Introduction
Studies exploring the use of affordable AI agents such as chatbots and conversational agents for second language (L2) learning have been increasing in recent years (Jeon, 2022; Lin & Mubarok, 2021; Underwood, 2017). AI agents provide students with ample opportunities to practice target languages in relatively stress-free learning environments, and particularly, offer personalized instruction for individual students in large classes (Dizon, 2020; Moussalli & Cardoso, 2020). Students have mostly reported that they enjoy and feel more relaxed about speaking to AI in L2 than to real humans (Wang et al., 2022; Tai & Chen, 2020). These benefits associated with AI agents offer promising solutions to intricate problems encountered in conventional L2 classrooms, such as inadequate speaking practice opportunities and limited class time (e.g., de Vries et al., 2015), unwillingness to speak (e.g., Tai & Chen, 2020), and lack of personalized feedback to every student (e.g., Luo, 2016).
Despite the benefits associated with AI-supported language learning, there is a noteworthy dearth of knowledge regarding how students interact with AI agents for language learning and what differences may exist among distinct types of students in human-AI interactions. Without such knowledge, the mechanism of AI-supported language learning remains a black box, that is, while students’ initial language competence at one end and their final competence at the other have been examined, what happens in between is largely unknown. Consequently, few insights can be gained to improve the design and implementation of AI agents in language learning as well as the pedagogy of AI-supported language learning.
To address this gap, this study investigated the human-AI interactions in an English as a Foreign Language (EFL) class involving 16 sixth-grade students in a primary school. An AI agent called the “AI coach” was used in this study. The AI coach was a virtual AI agent installed on mobile devices and was embodied as a female teacher. It was specifically created for EFL learning, supporting unlimited English practice and providing personalized feedback on students’ speaking.
Considering that prior studies in AI-supported language learning have anthropomorphized AI applications as peers, teachers, tutors, and teaching assistants for learners (Engwall & Lopes, 2020; Randall, 2019), students and AI may form a learning community where the social, cognitive, and teaching presences students perceive in interacting with AI can forge meaningful learning experiences and facilitate L2 acquisition (Jeon, 2022; Yu & Li, 2022). As such, theoretically, community of inquiry (CoI; Garrison et al., 2010; Garrison & Arbaugh, 2007) serves as a feasible framework for analyzing human-AI interactions to reveal how learning happens in AI-supported language learning (Wang et al., 2022; Yu & Li, 2022). Nonetheless, young learners may not necessarily be competent to use digital technologies, including AI, effectively for learning, though they are often perceived to be tech-savvy (Qi, 2019; Thompson, 2013). Substantial research (e.g., Niu et al., 2022; Ellis & Bliuc, 2019; Lindblom-Ylänne et al., 2019) has shown that the ways in which students approach online or offline learning play a crucial role in determining their learning experiences and performance. Students' approaches to learning (SAL) framework has been found to effectively explain why some students are more successful than others in school (Ellis & Bliuc, 2019). Though SAL have been widely applied in a variety of learning contexts (e.g., Biggs & Tang, 2011; Ellis & Bliuc, 2019; Hailikari & Parpala, 2014), it is still unknown how students’ approaches to AI-supported learning function in a human-AI community. Therefore, building on the combined frameworks of CoI and SAL, this study employed cluster analysis and epistemic network analysis (ENA) to provide insights into how distinct types of students interacted with the AI coach for EFL learning. Overall, this study sought to address the following two research questions:
(RQ1) How many distinct types of learners can be identified with respect to the AI usage?
(RQ2) How do distinct types of learners interact with AI for language learning and what are their differences in human-AI interactions, if any?
Section snippets
AI agents for language learning
AI agents in this study refer to entities that are capable of performing tasks related to intelligent beings through reasoning, learning, and expressing themselves, to a certain extent (Wang et al., 2022; Berendt et al., 2020). They are created in different forms (Randall, 2019), for instance, anthropomorphic (with human-like appearances), cartoon-like (with exaggerated cartoon features), and mechanomorphic (with machine-like features). Based on the visibility of AI agents, they can also be
Participants and research context
The present study was conducted in a 6th grade EFL class in an urban primary school in China, lasting approximately three months. The research context in this study was similar to Wang et al., 2022. The students started to learn English in Year 1 of primary school. Nonetheless, their English proficiency was normally at very basic levels due to limited exposure to authentic L2 environments and exam-oriented education. The AI coach (kouyu100.com) was purchased by the school to cope with the lack
Results
In this section, the results related to the cluster analysis were first reported, followed by the ENA results of different clusters of students’ retrospective interactions with the AI coach.
Discussion
Following the CoI and SAL frameworks, this study combined cluster analysis and epistemic network analysis to investigate how students of distinct types interacted with the AI coach for L2 learning and to reveal what matters in AI-supported language learning. Overall, three key findings can be derived.
Contributions and implications
This study contributes to AI-supported language learning in several ways. First, the present study advances the state-of-the-art understanding of how learners interact with AI technologies for language learning. Though AI has been considered useful and effective in improving learners' L2 competence and forging positive learning experiences, little is known regarding how human-AI interactions actually function for L2 learning. Though few studies (e.g., Lee & Jeon, 2022; Moussalli & Cardoso, 2020
Limitations and future research
The findings of this study should be interpreted with the following limitations in mind. First, this study involved a small number of participants studying EFL and was conducted in a primary school. Future research seeking to generalize the research findings can be conducted with more participants from diverse disciplines and school levels. Second, the present study mainly relied on two data sources: students' actual usage data and their reflection essays on their using experiences. To obtain
Credit author statement
Xinghua Wang: Conceptualization, Investigation, Methodology, Software, Formal analysis, Writing-original draft; Qian Liu: Conceptualization, Investigation, Formal analysis, Writing-original draft; Hui Pang: Resources, Data curation, Methodology, Writing-original draft, Writing-review & editing; Seng Chee Tan: Conceptualization, Methodology, Writing-review & editing. Jun Lei: Conceptualization, Methodology, Writing-review & editing. Matthew P. Wallace: Conceptualization, Methodology,
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
References (70)
- et al.
Engineering students' conceptions of and approaches to learning through discussions in face-to-face and online contexts
Learning and Instruction
(2008) - et al.
The first decade of the community of inquiry framework: A retrospective
The Internet and Higher Education
(2010) - et al.
Researching the community of inquiry framework: Review, issues, and future directions
The Internet and Higher Education
(2007) - et al.
Investigating learners' behaviors and discourse content in MOOC course reviews
Computers & Education
(2020) - et al.
Exploring engagement of users of Global Englishes in a community of inquiry
System
(2021) - et al.
Understanding the determinants of learner engagement in MOOCs: An adaptive structuration perspective
Computers & Education
(2020) The digital natives as learners: Technology use patterns and approaches to learning
Computers & Education
(2013)Teaching presence in telecollaboration. Keeping an open mind
System
(2017)- et al.
The impact of course duration on the development of a community of inquiry
Interactive Learning Environments
(2011) - et al.
The impact of social robotics on L2 learners' anxiety and attitude in English vocabulary acquisition
International Journal of Social Robotics
(2015)
Robotics and Weblog integration into EFL curriculum of middle and secondary schools: Exploratory study
Language Teaching Research
Using two-step cluster analysis and latent class cluster analysis to classify the cognitive heterogeneity of cross-diagnostic psychiatric inpatients
Frontiers in Psychology
AI in education: Learner choice and fundamental rights
Learning, Media and Technology
Social robots for language learning: A review
Review of Educational Research
What do inventories of students' learning processes really measure? A theoretical review and clarification
British Journal of Educational Psychology
Teaching for quality learning at university
Tablet‐based AR technology: Impacts on students' conceptions and approaches to learning mathematics according to their self‐efficacy
British Journal of Educational Technology
Exploring parents' conceptions of augmented reality learning and approaches to learning by augmented reality with their children
Journal of Educational Computing Research
Investigating college EFL learners' perceptions toward the use of Google Assistant for foreign language learning
Interactive Learning Environments
Evaluating intelligent personal assistants for L2 listening and speaking development
Language, Learning and Technology
Exploring new elements of the student approaches to learning framework: The role of online learning technologies in student learning
Active Learning in Higher Education
Interaction and collaboration in robot-assisted language learning for adults
Computer Assisted Language Learning
Interpretable MOOC recommendation: A multi-attention network for personalized learning behavior analysis
Internet Research
The role of the teacher and AI in education
Facilitating cognitive presence in online learning: Interaction is not enough
American Journal of Distance Education
The impact of training of university teachers on their teaching skills, their approach to teaching and the approach to learning of their students
Active Learning in Higher Education
What impedes or enhances my studying? The interrelation between approaches to learning, factors influencing study progress and earned credits
Teaching in Higher Education
Shadowing: Who benefits and how? Uncovering a booming EFL teaching technique for listening comprehension
Language Teaching Research
Learner control and error correction in ICALL: Browsers, peekers, and adamants
Calico Journal
Errors and intelligence in computer-assisted language learning: Parsers and pedagogues
Investigating the relationships between approaches to learning, learner identities and academic achievement in higher education
Higher Education
Learning behavior analysis of a ubiquitous situated reflective learning system with application to life science and technology teaching
Educational Technology & Society
Exploring AI chatbot affordances in the EFL classroom: Young learners' experiences and perspectives
Computer Assisted Language Learning
Finding groups in data: An introduction to cluster analysis
A comparison of three clustering methods for finding subgroups in MRI, SMS or clinical data: SPSS TwoStep cluster analysis, latent gold and SNOB
BMC Medical Research Methodology
Cited by (31)
Enhancing academic performance of business students using generative AI: An interactive-constructive-active-passive (ICAP) self-determination perspective
2024, International Journal of Management EducationA systematic review of AI education in K-12 classrooms from 2018 to 2023: Topics, strategies, and learning outcomes
2024, Computers and Education: Artificial IntelligenceChatGPT effects on cognitive skills of undergraduate students: Receiving instant responses from AI-based conversational large language models (LLMs)
2024, Computers and Education: Artificial Intelligence
- 1
Xinghua Wang and Qian Liu contribute equally to this study and share first authorship.