Patterns of use of an agent-based model and a system dynamics model: The application of patterns of use and the impacts on learning outcomes
Introduction
Buckley et al. (2004) suggest that use of simulation models, such as deciding which perceptual cues to attend to, deciding how to interact with the representation and monitoring and evaluating the results of those interactions, are important metacognitive processes that play an important part in model-based learning. If this is the case, learning outcomes should depend, amongst other factors, on the ways in which students interrogate simulations, although there are very few studies that investigate the use of simulation models and compare learning outcomes (Kennedy and Judd, 2007, Levy et al., 2004, Moxnes, 1998). Levy and Wilensky (2005) investigated the patterns of use of an agent-based model. They identified three distinct strategies for interrogating the model. In the study reported in this paper, video screen shots of students’ use of an agent-based model, a system dynamics model, and both combined were analysed in order to meet four aims: (1) to determine whether Levy and Wilensky’s strategies could be applied to other agent-based models, and to a system dynamics model; (2) to investigate the relationships between these strategies and general measures of use of the models to help to further define the classification system; (3) to investigate relationships between prior knowledge and the strategy chosen; and (4) to determine whether the strategy used was related to the learning outcomes.
The following literature review will outline the use of models for science and environmental education, provide details of the use of agent-based and system dynamics models, and discuss Levy and Wilensky’s strategies in more detail. The methods, including the design of the models, the experimental procedure and the instruments used to assess prior knowledge and learning outcomes, will also be explained. The results and discussion will follow.
Models are representations of ideas, objects, events, processes or systems (Gilbert & Boulter, 1998), and are generally simplifications of reality (Coyle, 2000, Jonassen, 2000). Computer-based models allow complex systems to be represented efficiently and constructed in a relatively short amount of time. Technology can be used for learning by modelling (Rohr and Reimann, 1998, Woolsey and Bellamy, 1997) and learning with models (Milrad, Spector, & Davidsen, 2003) to allow improved understanding of complex, dynamic systems. Many authors (see for example Stylianidou, Boohan, & Ogborn, 2004 refer to these types of learning as exploratory learning activities (students are able to explore pre-existing models) and expressive learning activities (students create their own models). A further step is for students to critique other students’ models (Gobert & Pallant, 2004). Understanding scientific models is quite different from the ability to reason with scientific models (Gobert & Pallant, 2004).
A system dynamics model was one of the types of model investigated in this study. “System Dynamics is a methodology for analysing complex systems and problems with the aid of computer simulation software” (Alessi, 2000, p. 1) and includes cause and effect relationships, time delays and feedback loops. Systems can be represented by causal loop diagrams and by stock and flow diagrams, and in this study, students were given access to the stock and flow diagram (see Fig. 1).
Although much is written of the value of system dynamics modelling in education in schools, very little empirical data exists to confirm this (Doyle et al., 1998, Stratford et al., 1998). Instead, case studies and anecdotal accounts from teachers who have used such models are available on the web (see for example Guthrie and Fisher, 1999, Ragan, 1999, Verona et al., 2001). The Core models project was a large scale implementation of system dynamics modelling in a number of schools, and evaluation of the project, while focusing on teacher support, did find that while students did improve understanding of the scientific concepts underlying modelling, they did not improve their ability to interpret the models (Maryland Virtual High School, 2001). Studies make recommendations as to how to teach system dynamics modelling (Schaffernicht, 2006, Stuntz, 2000), or how to incorporate it into a class (Draper, 1993).
Students do have trouble understanding complex systems using system dynamics models. One study found that the majority of participants in four separate studies (167 subjects in total) had biased views of the dynamics of the environmental system that they were examining which suggested that they were using a static (rather than a dynamic) mental model (Moxnes, 2000). Another study found that graduate students had very poor understanding of the processes involved in climate change, a common misconception was that stabilising emissions would “fix the problem”, showing a poor understanding of dynamic processes (Sterman & Booth Sweeny, 2002). While a number of studies have investigated learners’ inability to correctly model a natural resource problem (Booth Sweeney and Sterman, 2000, Diehl and Sterman, 1995, Moxnes, 2004), Kainz and Ossimitz (2002) determined that students did not have difficulties in determining between stocks and flows, and instead found it difficult to represent this information in a flow chart, and similarly that students were better able to interpret information from a table than from a graph. This suggests that perhaps the representational affordance of the system dynamics model, or the way in which the assessment is asked, needs to be examined.
An agent-based model is the other type of model that was investigated in this study. In agent-based modelling the focus is on the interaction between the agents, and their environment. An agent is defined as an object that controls its own behaviour, and could be individuals of a species, individuals at a particular stage in the life cycle (a cohort), or a group of individuals that can be considered identical (Ginot, Le Page, & Souissi, 2002). The rules that apply to the agents determine the behaviour of the whole system, called emergence. By laying down the rules for the agents and the system, behaviour may emerge that would otherwise not have been predicted (Bousquet and Le Page, 2004, Ginot et al., 2002, Parrott and Kok, 2001, Schieritz, 2002).
When used in education, agent-based models allow students to explore the relationship between the agents’ rules of behaviour and the patterns that emerge (Stieff & Wilensky, 2003). Students are able to make predictions and test them by exploring model outcomes as they manipulate variables (Stieff & Wilensky, 2003). The use of agent-based models in education “narrows the gap” between school biology and research biology (Wilensky & Reisman, 2006). The main advantage of using agent-based models is that students are able to employ their knowledge of the behaviour of individuals in the construction of theories about the behaviour of populations (Wilensky & Reisman, 2006). Agent-based models are able to be quite realistic. Using a realistic computer simulation can be motivating for students because they are entertaining and evocative (Goldstone & Son, 2005).
Levy and Wilensky (2005) investigated the patterns in how students used an agent-based model to learn about Chemistry. The four main statistics measured were successive settings in running the model, observation time, the average time between actions, and the number of runs. Students were engaged in a relatively ‘open’ activity (only able to change one setting, but to whatever value they chose), and Levy and Wilensky were able to identify three distinct exploration patterns: Straight To The Point, Homing In, and Oscillating.
The Straight To The Point strategy had a shorter overall observation time, but longer observation time per run; longer time between actions; and fewer runs. Levy and Wilensky identified this as an efficient mode to use the model, which may allow students to develop a deeper understanding of each state of the model. However, using a Straight To The Point strategy may mean that learners miss critical settings or transitions that would have been discovered by a wider range of values chosen. These critical settings are an important part of understanding a system from using a model (Lowe, 1993).
Students who used the Homing In strategy exhibited a shorter overall observation time, and shorter observation time per run; shorter time between actions; and more runs. These students were identified by the authors as “click happy”. The third strategy identified was called Oscillating, which consisted of a longer overall observation time, but shorter observation time per run; shorter time between actions; and an intermediate number of runs. Both the Homing In and Oscillating strategies involved speedy model changes and short observations, which implies that students were not able to detect and generalise complex relationships between variables. However, the many states of the model means that students examined many aspects of the model’s behaviour and students were more likely to detect a critical setting. Of the two, the Homing In strategy is more planned, whereas students would struggle to keep the previous state in mind for comparison when using the Oscillating strategy.
Strategies for interrogating models have been discussed, in some respect, in the system dynamics modelling literature; however no formal classification system has been developed. Moxnes (1998) discuses the combination of mental models and analysis. In terms of the analysis, he discusses a trial-and-error heuristic, a consistent analysis, and a gradient search, which can be loosely mapped to the Oscillating, Straight To The Point, and Homing In strategies noted by Levy and Wilensky. However, there was no further investigation into the links between the strategies and the decision-making and mental model development. Moxnes (1998) also investigated the use of explanatory features such as additional information that scaffolded students in how to model the system. Students were able to utilise the extra information given about growth, but not about the stock.
Another issue with regards to the use of the models is the role of prior knowledge in the use of models. The only study to specifically investigate the role of prior knowledge in the use of the model was the Levy and Wilensky (Levy & Wilensky, 2005) study discussed in some detail above. They suggested that prior knowledge about the domain may shorten the exploration time, resulting in a student focusing on a few key settings, such as the Straight To The Point strategy.
Section snippets
Use of the models
The system dynamics model was constructed in Stella™ (see Thompson & Reimann, 2007 for a more detailed description) and can be seen in Fig. 1 below. The user interface of the system dynamics model contained three screens analysed in this paper: the information screen, containing the text information describing the system; the explore screen, which allowed students to explore the model “step-by-step” using Stella’s™ storytelling feature (isee Systems., 2007, chap. 1), or in full which provided
Application of the strategies of use
Table 2 shows that in the ABM group, only two patterns were identified. Two students used the Straight To The Point strategy; the remaining students used the Oscillating strategy.
Students’ patterns of use of the model in the SDM group were recorded and classified using Levy and Wilensky’s method. Table 3 shows that in the SDM group, all three patterns were identified. Three students used the Straight To The Point strategy; two students used the Oscillating strategy; and two students used the
Classification of strategies used
All strategies were used by students in the SDM group and the SDM and ABM group, and both the Oscillating and Straight To The Point strategies were used by students in the ABM group. An additional strategy was identified: Oscillating over time. This strategy is a combination of the Homing In and the Oscillating strategy. There is an overall Oscillating pattern, but with changes made between the extremes. This strategy is expected to have similar advantages to the Homing In strategy and the
Conclusion
In conclusion, the classification scheme was able to be applied to the use of a system dynamics model; and an extra classification and an additional criterion were suggested. Prior knowledge had an effect on the strategy used to change specific variables. The proportion of time spent on the information screen was important for students who used the Straight To The Point strategy. Despite this more informed approach to the Straight To The Point strategy, students who used it overall, and to
Acknowledgement
We would like to thank the students and teachers involved who participated. Thank you too to Chloe (Yun Cai) for her help in coding the videos.
References (40)
- et al.
Multi-agent simulations and ecosystem management: A review
Ecological Modelling
(2004) - et al.
Effects of feedback complexity on dynamic decision making
Organizational Behaviour and Human Decision Processes
(1995) - et al.
A multi-agents architecture to enhance end-user individual-based modelling
Ecological Modelling
(2002) - et al.
Expectations and reality: Evaluating patterns of learning behaviour using audit trails
Computers and Education
(2007) Constructing a mental representation from an abstract technical diagram
Learning and Instruction
(1993)Overexploitation of renewable resources: The role of misperceptions
Journal of Economic Behaviour & Organization
(1998)- et al.
A generic primary producer model for use in ecosystem simulation
Ecological Modelling
(2001) - Alessi, S. (2000). The application of system dynamics in elementary and secondary school curricular....
- et al.
Bathtub dynamics: Initial results of a systems thinking inventory
System Dynamics Review
(2000) - et al.
Model-based teaching and learning with BioLogica: What do they learn? How do they learn? How do we know?
Journal of Science Education and Technology
(2004)
Qualitative and quantitative modelling in system dynamics: Some research questions
System Dynamics Review
A proposed sequence for developing systems thinking in a grades 4–12 curriculum
System Dynamics Review
Discovering statistics using SPSS
Learning science through models and modelling
Fostering Students’ epistemologies of models via authentic model-based tasks
Journal of Science Education and Technology
The transfer of scientific principles using concrete and idealized simulations
The Journal of the Learning Sciences
Computers as mind tools for schools
Cited by (25)
Teachers’ use and adaptation of a model-based climate curriculum: a three-year longitudinal study
2024, Journal of Geoscience EducationFacilitated model-based reasoning in immersive virtual reality: Meaning-making and embodied interactions with dynamic processes
2023, International Journal of Computer-Supported Collaborative LearningWorking Together: Integrating Computational Modeling Approaches to Investigate Complex Phenomena
2021, Journal of Science Education and TechnologyModel-Based Reasoning with Immersive VR Simulations: Patterns of Use Grounded in Time and 3D Space
2021, Proceedings of International Conference of the Learning Sciences, ICLS