Skip to main content
SpringerLink
Log in
Book cover

International Conference on Artificial Intelligence in Education

AIED 2018: Artificial Intelligence in Education pp 253–266Cite as

How to Use Simulation in the Design and Evaluation of Learning Environments with Self-directed Longer-Term Learners

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10947))

Abstract

Designing, developing, and evaluating interactive and adaptive learning environments requires significant investment of financial and human resources. This is especially the case when evaluating the impact of learning environments aimed at supporting self-directed longer-term learners, environments of increasing interest to AIED as the field moves into lifelong learning where mentorship is key. In this paper we propose the use of simulation to help in both the design and evaluation of such environments. As a case study, we have built a simulated university doctoral program (SimDoc) with simulated doctoral students and supervisors (mentors). To make sure SimDoc replicates observed data from a real-world environment as closely as possible, we informed and calibrated the simulation model with data from an actual doctoral program, as well as drawing on various empirical studies of graduate students and supervisors. Next, we used the calibrated simulation model to explore the effect of varying research group sizes of learners and supervisors’ mentoring workload on students’ completion rates and time-to-completion. Our main goal is to provide insight into how to build simulations of environments that support self-directed longer-term learners through a case study of one such simulation, thus further demonstrating the importance of simulation in AIED.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://www.usask.ca/ict/services/ent-business-intelligence/university-data-warehouse.php, last accessed on February 06, 2018.

  2. 2.

    https://grad.usask.ca/programs/find-a-program.php last accessed on February 06, 2018.

  3. 3.

    http://artsandscience.usask.ca/psychology/department/gradteaching.php last accessed on February 06, 2018.

  4. 4.

    https://www.grad.ubc.ca/campus-community/life-grad-student-ubc/what-expect last accessed on February 06, 2018.

References

  1. Bull, S., Kay, J.: SMILI: a framework for interfaces to learning data in open learner models, learning analytics and related fields. Int. J. Artif. Intell. Educ. 26(1), 293–331 (2016)

    Article  Google Scholar 

  2. Lane, H.C., McCalla, G., Looi, C.-K., Bull, S.: The next 25 years: how advanced interactive learning technologies will change the world. Int. J. Artif. Intell. Educ. 26(1), 539–543 (2016)

    Article  Google Scholar 

  3. Erickson, G., Frost, S., Bateman, S., McCalla, G.: Using the ecological approach to create simulations of learning environments. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 411–420. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_42

    Chapter  Google Scholar 

  4. VanLehn, K., Ohlsson, S., Nason, R.: Applications of simulated students: an exploration. Int. J. Artif. Intell. Educ. 5(2), 1–42 (1994)

    Google Scholar 

  5. Koedinger, K.R., Matsuda, N., MacLellan, C.J., McLaughlin, E.A.: Methods for evaluating simulated learners: examples from SimStudent. In: The Proceedings of the 2nd Workshop on Simulated Learners at the 17th International Conference on Artificial Intelligence in Education, vol. 5, pp. 45–54 (2015)

    Google Scholar 

  6. Champaign, J., Cohen, R.: A multiagent, ecological approach to content sequencing. In: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems - AAMAS 2010, pp. 10–14 (2010)

    Google Scholar 

  7. Frost, S., McCalla, G.: Exploring through simulation an instructional planner for dynamic open-ended learning environments. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M.F. (eds.) AIED 2015. LNCS (LNAI), vol. 9112, pp. 578–581. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19773-9_66

    Chapter  Google Scholar 

  8. González-Brenes, J.P., Huang, Y.: Using data from real and simulated learners to evaluate adaptive tutoring systems. In: The Proceedings of the 2nd Workshop on Simulated Learners at the 17th International Conference on Artificial Intelligence in Education, vol. 5, pp. 31–34 (2015)

    Google Scholar 

  9. Pezzullo, L.G., Wiggins, J.B., Frankosky, M.H., Min, W., Boyer, K.E., Mott, B.W., Wiebe, E.N., Lester, J.C.: “Thanks Alisha, keep in touch”: gender effects and engagement with virtual learning companions. In: André, E., Baker, R., Hu, X., Rodrigo, Ma.M.T., du Boulay, B. (eds.) AIED 2017. LNCS (LNAI), vol. 10331, pp. 299–310. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61425-0_25

    Chapter  Google Scholar 

  10. Pareto, L.: A teachable agent game engaging primary school children to learn arithmetic concepts and reasoning. Int. J. Artif. Intell. Educ. 24(3), 251–283 (2014)

    Article  Google Scholar 

  11. Palmqvist, L., Kirkegaard, C., Silvervarg, A., Haake, M., Gulz, A.: The relationship between working memory capacity and students’ behaviour in a teachable agent-based software. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M.F. (eds.) AIED 2015. LNCS (LNAI), vol. 9112, pp. 670–673. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19773-9_88

    Chapter  Google Scholar 

  12. Sjödén, B., Tärning, B., Pareto, L., Gulz, A.: Transferring teaching to testing – an unexplored aspect of teachable agents. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS (LNAI), vol. 6738, pp. 337–344. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21869-9_44

    Chapter  Google Scholar 

  13. Matsuda, N., Yarzebinski, E., Keiser, V., Raizada, R., Stylianides, G.J., Cohen, W.W., Koedinger, K.R.: Learning by teaching SimStudent – an initial classroom baseline study comparing with cognitive tutor. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS (LNAI), vol. 6738, pp. 213–221. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21869-9_29

    Chapter  Google Scholar 

  14. Matsuda, N., Cohen, W.W., Koedinger, K.R.: Teaching the teacher: tutoring Simstudent leads to more effective cognitive tutor authoring. Int. J. Artif. Intell. Educ. 25(1), 1–34 (2015)

    Article  Google Scholar 

  15. Pardos, Z.A., Yudelson, M.V.: Towards moment of learning accuracy. In: The Proceedings of the 1st Workshop on Simulated Learners at the 16th International Conference on Artificial Intelligence in Education, vol. 7, pp. 61–70 (2013)

    Google Scholar 

  16. Wang, Y., Heffernan, N.: A comparison of two different methods to individualize students and skills. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 836–839. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_125

    Chapter  Google Scholar 

  17. Folsom-Kovarik, J.T., Wray, R.E., Hamel, L.: Adaptive assessment in an instructor-mediated system. In: Lane, H.Chad, Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 571–574. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_61

    Chapter  Google Scholar 

  18. Ueno, M.: Adaptive testing based on bayesian decision theory. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 712–716. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_95

    Chapter  Google Scholar 

  19. Faghihi, U., Fournier-Viger, P., Nkambou, R.: A cognitive tutoring agent with automatic reasoning capabilities. In: The Proceedings of the 24th International Florida Artificial Intelligence Research Society Conference, pp. 448–449 (2011)

    Google Scholar 

  20. Maclellan, C.J., Matsuda, N., Koedinger, K.R.: Toward a reflective SimStudent: using experience to avoid generalization errors. In: The Proceedings of the 1st Workshop on Simulated Learners at the 16th International Conference on Artificial Intelligence in Education, pp. 51–60 (2013)

    Google Scholar 

  21. Lelei, D.E.K., McCalla, G.: Exploring the issues in simulating a semi-structured learning environment: the SimGrad doctoral program design. In: The Proceedings of the 2nd Workshop on Simulated Learners at the 17th International Conference on Artificial Intelligence in Education, vol. 5, pp. 11–20 (2015)

    Google Scholar 

  22. Groenvynck, H., Vandevelde, K., Van Rossem, R.: The Ph.D. track: who succeeds, who drops out? Res. Eval. 22(4), 199–209 (2013)

    Article  Google Scholar 

  23. Gatfield, T.: An investigation into PhD supervisory management styles: development of a dynamic conceptual model and its managerial implications. J. High. Educ. Policy Manag. 27(3), 311–325 (2005)

    Article  Google Scholar 

  24. Heath, T.: A quantitative analysis of PhD students’ views of supervision. High. Educ. Res. Dev. 21, 37–41 (2002)

    Article  Google Scholar 

  25. Torrance, M., Thomas, G.V., Robinson, E.J.: The writing strategies of graduate research students in the social sciences. High. Educ. 27(3), 379–392 (1994)

    Article  Google Scholar 

  26. Mainhard, T., van der Rijst, R., van Tartwijk, J., Wubbels, T.: A model for the supervisor-doctoral student relationship. High. Educ. 58(3), 359–373 (2009)

    Article  Google Scholar 

  27. Esteva, M., Rodríguez-Aguilar, J.-A., Sierra, C., Garcia, P., Arcos, J.L.: On the formal specification of electronic institutions. In: Dignum, F., Sierra, C. (eds.) Agent Mediated Electronic Commerce. LNCS (LNAI), vol. 1991, pp. 126–147. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44682-6_8

    Chapter  Google Scholar 

  28. Berendsen, H.J.C., Van Der Spoel, D., Van Drunen, R.: GROMACS: a message-passing parallel molecular dynamics implementation. Comput. Phys. Commun. 91(1), 43–56 (1995)

    Article  Google Scholar 

  29. Lavelle, E., Bushrow, K.: Writing approaches of graduate students. Educ. Psychol. 27(6), 807–822 (2007)

    Article  Google Scholar 

  30. Seagram, B.C., Gould, J., Pyke, S.W.: An investigation of gender and other variables on time to completion of doctoral degrees. Res. High. Educ. 39(3), 319–335 (1998)

    Article  Google Scholar 

  31. Pelánek, R., Jarušek, P.: Student modeling based on problem solving times. Int. J. Artif. Intell. Educ. 25(4), 493–519 (2015)

    Article  Google Scholar 

  32. Axelrod, R.: Advancing the art of simulation in the social sciences. In: Conte, R., Hegselmann, R., Terna, P. (eds.) The Proceedings of the 18th European Meeting on Cybernetics and Systems Research, pp. 1–13. Springer, Heidelberg (2006). https://doi.org/10.1007/978-3-662-03366-1_2

  33. Gilbert, N., Terna, P.: How to build and use agent-based models in social science. Mind Soc. 1(1), 57–72 (2000)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the University of Saskatchewan University Data Warehouse team for allowing us to access their dataset, and the Natural Sciences and Engineering Research Council of Canada (NSERC) for funding our research.

Author information

Authors and Affiliations

  1. ARIES Laboratory, Department of Computer Science, University of Saskatchewan, Saskatoon, Canada

    David Edgar Kiprop Lelei & Gordon McCalla

Authors
  1. David Edgar Kiprop Lelei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gordon McCalla
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to David Edgar Kiprop Lelei or Gordon McCalla .

Editor information

Editors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, USA

    Carolyn Penstein Rosé

  2. University of Technology, Sydney, NSW, Australia

    Roberto Martínez-Maldonado

  3. University of Duisburg-Essen, Duisburg, Germany

    H. Ulrich Hoppe

  4. UCL Institute of Education, London, UK

    Rose Luckin

  5. UCL Institute of Education, London, UK

    Manolis Mavrikis

  6. UCL Institute of Education, London, UK

    Kaska Porayska-Pomsta

  7. Carnegie Mellon University, Pittsburgh, PA, USA

    Bruce McLaren

  8. University of Sussex, Brighton, UK

    Benedict du Boulay

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lelei, D.E.K., McCalla, G. (2018). How to Use Simulation in the Design and Evaluation of Learning Environments with Self-directed Longer-Term Learners. In: Penstein Rosé, C., et al. Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science(), vol 10947. Springer, Cham. https://doi.org/10.1007/978-3-319-93843-1_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-93843-1_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-93842-4

  • Online ISBN: 978-3-319-93843-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation