Elsevier

Computers & Education

Volume 160, January 2021, 104029
Computers & Education

The affordances and limitations of collaborative science simulations: The analysis from multiple evidences

https://doi.org/10.1016/j.compedu.2020.104029Get rights and content

Highlights

  • Recent Web techniques have made collaborative simulations feasible.

  • Students learning with collaborative simulation displayed lower levels of individual attention.

  • Their attentions switched among different AOIs.

  • They demonstrated more constructive discourse threads.

  • Students are more likely to engage in effective collaborative learning.

Abstract

Recent advancements in Web and HTML5 techniques have made collaborative simulations able to support a shared workspace and more synchronous collaboration in a single simulation. However, previous studies show divergent findings regarding the effects of the new features of collaborative simulations. To better understand the affordances and limitations of collaborative simulations, this study analyzed how 64 students learned in two different collaborative learning settings: individual-based simulation and collaborative simulation. To better reveal how students attended to different parts of the simulation, the researchers collected eye movement data and analyzed students’ discourse, perceptions of collaboration and learning performance. The findings revealed that the two groups demonstrated similar levels of learning gains. However, the students who used the collaborative simulation displayed lower levels and less frequency of individual attention to the simulation than those using the individual-based simulation. However, the joint attention and discourse analysis suggests that students learning with the collaborative simulation are more likely to engage in effective collaborative learning while demonstrating more constructive discourse threads than those using the individual-based simulation. The findings support that the collaborative simulations transformed the learning experience into a cohesive collaborative learning process. This study also found that the relationship between the joint attention and the discourse features as well as the perceived collaborative experience are not decisive but are influenced by the features of the collaborative learning environment. This article discusses the implications of the use of collaborative simulations and eye movement for supporting collaborative learning, and addresses the direction for future studies.

Introduction

Computer simulations provide opportunities for students to virtually interact with science phenomena (Sulisworo, Handayani, & Kusumaningtyas, 2019). For example, computer simulations available on the PhET, molecular workbench, or CoSci online platforms are especially helpful for assisting students in modeling abstract and invisible science phenomena as they support visualization and manipulation functions (Wang, Wu, & Hsu, 2017). Researchers and educators have identified the positive effects of science simulations on science learning. Science simulations allow students to experience the science inquiry process (Wen et al., 2018), facilitate knowledge integration processes (Taub, Armoni, Bagno, & Ben-Ari, 2015) and help students learn analytical skills (Pedaste & Sarapuu, 2014). Researchers and educators extensively apply computer simulations to motivate students, increase their self-reflection (Mawhirter & Garofalo, 2016), and help them develop critical thinking skills (Bell & Loon, 2015). Previous studies have also shown that collaborative application of computer simulations improves students’ learning performance. Students who collaboratively learned via the simulation learned better than those who used the simulation on their own (Hao, Liu, von Davier, & Kyllonen, 2015; Ke & Carafano, 2016). However, the science simulations applied in the literature are individual-based simulations which mainly support individual students exploring a science phenomenon on an individual basis without supporting collaborative features such as synchronization of simulation status and a shared workspace.

Collaborative simulations (Care & Griffin, 2014; Chang, Chang, Liu, et al., 2017; Chang, Chang, Chiu, et al., 2017; de la Torre, Heradio, Dormido, & Jara, 2013; Jara, Candelas, Torres, Dormido, & Esquembre, 2012) which allow multiple students to jointly and synchronously operate a science phenomenon have been applied to support collaborative scientific modeling activities and collaborative problem solving activities. Such development of science simulations relies mainly on the advancements in Web and HTML5 techniques that can support higher levels of data communication and synchronization. In the collaborative simulations, students’ actions are closely synchronized. Together with the verbal communication channel such as text-based chatroom or voice communication tools, the collaborative simulations create a dual interaction space (Jermann & Dillenbourg, 2008) that can possibly lead to higher levels of collaboration effect. However, empirical studies have yielded divergent findings regarding the effect of the collaborative simulations (Chang et al., 2017a, 2017b). This study thus attempted to analyze the collaborative learning supported by the collaborative simulation from divergent data sources. It is hoped that this study can obtain a better understanding of the affordances and limitations of collaborative science simulations through the detailed analysis.

Shared representations such as simulations are a mediator to facilitate learners' generation of productive conversations (Suthers, 2006). Due to their visible feature, simulations visualize science concepts explicitly to help learners co-construct science knowledge and negotiate with each other (Jeong & Hmelo-Silver, 2016). Collaborative simulations can afford a shared workplace, which is one of the crucial features to support team members in constructing shared understanding (Sun, Yuan, Rosson, Wu, & Carroll, 2017). The learners who collaborate with partners but who work in independent workplaces tend to engage in individual work rather than in collaborative work (Scott, Graham, Wallace, Hancock, & Nacenta, 2015). The lack of a shared workspace may lead to a significant decrease in shared visual attention and activity awareness (Chung, Lee, & Liu, 2013). This is partially because the shared workspace increases the visual awareness of the problem context. Therefore, the shared workspace has a significant impact on the team's discussion, and helps team members to achieve a better shared understanding of the problem, especially in remote collaborations (Müller, Rädle, & Reiterer, 2017). In this sense, collaborative simulations have the potential to improve the collaborative learning process because they can act as the shared workspace in which all the operations of the target science phenomenon from all participating members are visible to all members.

Researchers have attempted to enhance the features of computer simulations to better support collaborative science learning. For instance, the collaborative inquiry environment, Co-Lab, combined modeling and collaboration to support collaborative inquiry-based learning (van Joolingen, de Jong, Lazonder, Savelsbergh, & Manlove, 2005). Findings of previous studies show that it is a valid way to support students to complete complex modeling tasks (Bravo, van Joolingen, & de Jong, 2009). Other studies (e.g., de la Torre et al., 2013; Jara et al., 2012) proposed the notion of shared virtual laboratories. This form of collaborative simulation supports a group of students in jointly conducting science experiments. More specifically, such simulations synchronized all the equipment and the states of all equipment in the shared virtual laboratory among all participating students. Previous studies confirmed that the collaborative simulations improved students’ engagement and learning gains (de la Torre et al., 2013).

Another line of research in this direction is the enforcement of personal accountability on the operations of simulations (Care & Griffin, 2014; Chang et al., 2017a, 2017b). While participating students can jointly participate in a shared simulation session, the collaborative simulations assign participating students different accountability. Such a design urges students to closely coordinate with each other to operate the simulation to solve a shared problem. The students manipulated the simulation together according to their responsibility to collaborate with team members to conduct experiments synchronously. However, the study by (Chang, Chang, Liu, et al., 2017) identified a negative effect of the collaborative simulation. While students learning with individual-based simulation achieved significant improvement in their conceptual understanding, the students learning with the collaborative simulation did not. Therefore, further investigations of the cause of these conflicting findings is necessary.

The collaborative process and the cognitive loads caused by different simulations can explain the conflicting empirical findings. Cognitive load theory (Kirschner, Sweller, Kirschner, & Zambrano, 2018) considers the mutual cognitive interdependence principle as the foundation of collaborative learning. The principle asserts that collective working memory can be created through communicating and coordinating to generate a better collective knowledge structure. Furthermore, collaboration may reduce the extraneous load as the complex problem elements can be distributed among individual working memories, and effective collaboration also works as a scaffold to guide individuals’ knowledge acquisition processes. However, according to the cognitive theory, both the learning task and the collaboration produce extraneous load. The collaboration may also interfere with the individual learning process (Chang, Chang, Liu, et al., 2017). When the collaboration task becomes complex, the cognitive load associated with the learning task will also increase, which in turn will have a negative impact on learning (Beserra, Nussbaum, Oteo, & Martin, 2014). The conflicting empirical findings call for further investigations on the detailed process at both the individual and collaborative learning levels to understand how the collaborative simulations affect the learning process.

Due to the rapid technological development, researchers gradually applied eye movement analyses (Rayner, 1998) in the educational and learning field (Lai et al., 2013), particularly in the digital learning research domain (Alemdag & Cagiltay, 2018; Scheiter et al., 2019; Yang et al., 2018). To understand the detailed process when students learn with simulations, empirical studies have applied eye-tracking techniques to capture and analyze student eye movement on the simulations. The study by She and Chen (2009) confirmed that students showed different attention patterns to simulations with interactive features and pure animation. The eye movement data are helpful for displaying how students pay attention to different elements of the learning content during the learning process, and thus can help improve the content and pedagogical design. Results of the study by Kardan and Conati (2012) have shown that certain eye movement patterns can predict learning outcomes, and may reveal different levels of cognitive development through the use of simulation (O’Keefe, Letourneau, Homer, Schwartz, & Plass, 2014). Recently, Chiou, Hsu, and Tsai (2019) analyzed individuals' eye fixation data dynamically along with log data to explore deeply how eighth graders interact visually and manually with a physics simulation embedded with scientific inquiry guidance. They found that just following the guidance manually is not sufficient for successful inquiry learning, and students' visual attention paid to the relevant area in simulation may be critical for successful simulation-based inquiry learning.

The above studies show how eye movement analysis can be helpful in displaying individual students' attention to simulations. However, collaborative learning with simulations involves two cognitive systems interacting with each other to obtain a shared view. “Collaboration is a coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem” (Roschelle & Teasley, 1995, p. 70). Schneider and Pea (2013) thus suggested an operational method to study collaborative learning using eye movement analysis which reveals the tendency of two collaborative partners to focus on a common reference. This method relies on the close relation between human attention and discourse, indicating that individual attention is influenced by an individual's goals and the conversation discourse (Fang, Chai, & Ferreira, 2009). As eye movements may reveal individuals' information processing behaviors, the study of how the two eye movement patterns demonstrated by the two collaborating partners can reveal how they collaborate with each other to process the information. Empirical studies have also indicated the close relation between the dual gaze movement and collaboration quality as well as learning gains (Othlinghaus-Wulhorst, Jedich, Hoppe, & Harrer, 2018, pp. 185–197; Schneider et al., 2016a). Therefore, this study analyzed both the individual and joint attention of students when they were learning together with a simulation.

Numerous studies have applied eye movement analysis to depict the detailed process of learning with images (Schneider & Pea, 2013, 2014, pp. 138–144), tangible learning environments (Schneider et al., 2016a, 2016b), intelligent tutoring systems (Olsen, Sharma, Aleven, & Rummel, 2018), and MOOCs (Sharma, Jermann, Nüssli, & Dillenbourg, 2013). These studies investigated collaborative learning from diverse dimensions including joint visual attention, learning gains, perception of collaboration, and discourse features. These studies confirmed that certain technological designs can enhance the collaboration process. For instance, Schneider and Pea (2013) found that the awareness of the collaborating peers’ visual attention can enhance joint visual attention and improve the learning gains. This may be because student pairs who demonstrated frequent joint visual attention are more likely to conduct effective discussion (Olsen et al., 2018).

However, the literature also reveals inconsistent findings. The studies by Schneider and Pea (2014, pp. 138–144) and Schneider et al. (2016a) found a positive correlation between joint attention and learning gains. In contrast, there was no significant correlation between learning gains and joint visual attention in the study by Schneider et al. (2016b). Regarding the perception of collaboration, Schneider and Pea (2013) and Othlinghaus-Wulhorst et al. (2018) found that students’ joint visual attention was positively correlated with their collaboration quality, while the same correlation was not found in the study by Sharma et al. (2013).

The above empirical studies confirmed that the technological design may impact the collaborative learning process. In different technological designs, the collaboration process may differ, and thus students’ joint visual attention can also differ, leading to certain collaboration features in discourse, perception and learning gains. As collaborative simulations provide new technological features including synchronization of actions and a shared workspace for collaborating partners, the simulations may have an impact on the collaboration learning process. Therefore, this study investigated the collaborative learning from different data sources. The researchers collected data from 64 students engaged in collaborative learning with two different computer simulations, that is, individual-based simulation and collaborative simulation. This study analyzed multiple data sources including visual attention by eye-tracking techniques, discourse data, pre- and post-test achievement, and perceptions of the collaboration process to answer the following research questions:

RQ1: Do the two simulations (individual and collaborative simulations) have an impact on students’ individual and joint attention to the simulations?

RQ2: Do the two simulations have an impact on students' discourse to solve the problem? How is students’ joint visual attention correlated with their discourse features?

RQ3: Do the two simulations have an impact on perceived quality of collaboration? And how is students’ joint visual attention correlated with the perceived quality of collaboration?

RQ4: Do the students using the two simulations (individual and collaborative simulations) achieve different learning gains?

Section snippets

Participants

This study recruited students from a university in northern Taiwan via the university social media. Sixty-four students (38 females; 26 males) aged from 18 to 25 volunteered to join this study. They were majoring in various disciplines: nine in the college of liberal arts, eight in the college of science, 14 in the college of engineering, 21 in electrical engineering and computer science, and 12 in the school of management. The researchers paired these students to participate in the

Individual attention to the simulation

Regarding the whole learning task, the comparison of the individual attention indicators of the two groups is presented in Table 4. The results show that there is no significant difference in the Total Time Tracked (TTT) (t = 1.00, p = 0.32), and the Average Fixation Duration (AFD) (t = 0.98, p = 0.33). However, the two groups showed marginally significant difference in the Total Fixation Duration (TFD) (t = 1.86, p = 0.07) and Total Fixation Count (TFC) (t = 1.75, p = 0.09). In other words,

Discussion

In the present study, we attempted to investigate the affordances and limitations of two computer simulations: individual-based simulation and collaborative simulation, by comparing the participants’ eye movement, collaborative perceptions, discourse features and learning gains.

Conclusion

This present study provides evidence of the affordances and limitations of collaborative and individual-based simulations. Both the students using the collaborative simulation and individual-based simulation demonstrated significant learning gains. However, the students using collaborative simulation and the individual-based simulation revealed different attention patterns. While the students using the individual-based simulation are more likely to constantly pay attention to the area of

Author contribution

Chen-Chung Liu, Conceptualization, Methodology, Formal analysis, Writing - original draft, Writing - review & editing. I-Chen Hsieh, Conceptualization, Methodology, Software, Formal analysis. Cai-Ting Wen, Conceptualization, Software, Formal analysis, Writing - original draft, Writing - review & editing. Ming-Hua Chang, Software. Shih-Hsun Fan Chiang, Software. Meng-Jung Tsai, Methodology, Validation. Chia-Jung Chang, Methodology. Fu-Kwun Hwang, Software.

Acknowledgments

This research was partially funded by the Ministry of Science and Technology, Taiwan under contract numbers 109-2511-H-008 -006 -MY3, 108-2811-H-008 -506 -, 107-2511-H-008 -003 -MY3.This study also thanks to the eye-tracking techniques provided by MJ's CELL Research Team sponsored by MOST 106-2511-S-003-064-MY3 and the ‘Institute for Research Excellence in Learning Sciences’ of NTNU sponsored by the Ministry of Education (MOE) in Taiwan.

References (62)

  • N. Marwan et al.

    Nonlinear analysis of bivariate data with cross recurrence plots

    Physics Letters A

    (2002)
  • P.A. O'Keefe et al.

    Learning from multiple representations: An examination of fixation patterns in a science simulation

    Computers in Human Behavior

    (2014)
  • K. Scheiter et al.

    Adaptive multimedia: Using gaze-contingent instructional guidance to provide personalized processing support

    Computers & Education

    (2019)
  • H.C. She et al.

    The impact of multimedia effect on science learning: Evidence from eye movements

    Computers & Education

    (2009)
  • M. Staudte et al.

    Investigating joint attention mechanisms through spoken human–robot interaction

    Cognition

    (2011)
  • R. Taub et al.

    The effect of computer science on physics learning in a computational science environment

    Computers & Education

    (2015)
  • J.Y. Wang et al.

    Using mobile applications for learning: Effects of simulation design, visual-motor integration, and spatial ability on high school students' conceptual understanding

    Computers in Human Behavior

    (2017)
  • E. Care et al.

    An approach to assessment of collaborative problem solving

    Research and Practice in Technology Enhanced Learning

    (2014)
  • C.J. Chang et al.

    An analysis of collaborative problem‐solving activities mediated by individual‐based and collaborative computer simulations

    Journal of Computer Assisted Learning

    (2017)
  • G.-L. Chiou et al.

    Exploring how students interact with guidance in a physics simulation: Evidence from eye-movement and log data analyses. Interactive Learning Environments

  • M.T. Chi et al.

    The ICAP framework: Linking cognitive engagement to active learning outcomes

    Educational Psychologist

    (2014)
  • C.W. Chung et al.

    Investigating face‐to‐face peer interaction patterns in a collaborative web discovery task: The benefits of a shared display

    Journal of Computer Assisted Learning

    (2013)
  • T.P. Falcão et al.

    June). What have you done! the role of'interference'in tangible environments for supporting collaborative learning

  • R. Fang et al.

    November). Between linguistic attention and gaze fixations inmultimodal conversational interfaces

  • J. Hao et al.

    Assessing collaborative problem solving with simulation based tasks

    (2015)
  • J.A. Harsh et al.

    “Seeing” data like an expert: An eye-tracking study using graphical data representations

    CBE-Life Sciences Education

    (2019)
  • M. Hoegl et al.

    Teamwork quality and the success of innovative projects: A theoretical concept and empirical evidence

    Organization Science

    (2001)
  • C.Y. Hsu et al.

    A pilot study on developing and validating a fixation-based scaffolding learning system

  • J. Janssen et al.

    Multilevel analysis for the analysis of collaborative learning

    The international handbook of collaborative learning

    (2013)
  • C.A. Jara et al.

    Synchronous collaboration of virtual and remote laboratories

    Computer Applications in Engineering Education

    (2012)
  • H. Jeong et al.

    Seven affordances of computer-supported collaborative learning: How to support collaborative learning? How can technologies help?

    Educational Psychologist

    (2016)
  • Cited by (0)

    View full text