Skip to main content
Springer Nature Link
Log in

Involving Teachers in the Data-Driven Improvement of Intelligent Tutors: A Prototyping Study

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13916))

Included in the following conference series:

  • 4766 Accesses

Abstract

Several studies show that log data analysis can lead to effective redesign of intelligent tutoring systems (ITSs). However, teachers are seldom included in the data-driven redesign of ITS, despite their pedagogical content knowledge. Examining teachers’ possible contributions is valuable. To investigate what contributions teachers might make and whether (and how) data would be useful, we first built an interactive prototype tool for visualizing student log data, SolutionVis, based on needs identified in interviews with tutor authors. SolutionVis presents students’ problem-solving processes with an intelligent tutor, including meta-cognitive aspects (e.g., hint requests). We then conducted a within-subjects user study with eight teachers to compare teachers’ redesign suggestions obtained in three conditions: a baseline “no data” condition (where teachers examined just the tutor itself) and two “with data” conditions in which teachers worked with SolutionVis and with a list representation of student solutions, respectively. The results showed that teachers generated useful redesign ideas in all three conditions, that they viewed the availability of data (in both formats) as helpful and enabled them to generate a wider range of redesign suggestions, specifically with respect to hint design and feedback on gaming-the-system behaviors and struggle. The current work suggests potential benefits and ways of involving teachers in the data-driven improvement of ITSs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Grastable math (2022). https://activities.graspablemath.com/. Accessed 10 Sept 2022

  2. Cytoscape.js (2023). https://js.cytoscape.org/. Accessed 29 Apr 2023

  3. Aleven, V., Blankestijn, J., Lawrence, L., Nagashima, T., Taatgen, N.: A dashboard to support teachers during students’ self-paced AI-supported problem-solving practice. In: Hilliger, I., Muñoz-Merino, P.J., De Laet, T., Ortega-Arranz, A., Farrell, T. (eds.) EC-TEL 2022. LNCS, vol. 13450, pp. 16–30. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16290-9_2

    Chapter  Google Scholar 

  4. Aleven, V., Sewall, J.: The frequency of tutor behaviors: a case study. In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 396–401. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39583-8_47

    Chapter  Google Scholar 

  5. Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the system usability scale. IJHCI 24(6), 574–594 (2008)

    Google Scholar 

  6. Hartson, R., Pyla, P.S.: The UX Book: Process and Guidelines for Ensuring a Quality User Experience. Elsevier, Amsterdam (2012)

    Google Scholar 

  7. Heffernan, N.T., Heffernan, C.L.: The assistments ecosystem: building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. IJAIED 24(4), 470–497 (2014)

    MathSciNet  Google Scholar 

  8. Holstein, K., McLaren, B.M., Aleven, V.: Co-designing a real-time classroom orchestration tool to support teacher-AI complementarity. J. Learn. Anal. 6(2) (2019)

    Google Scholar 

  9. Huang, Y., et al.: A general multi-method approach to data-driven redesign of tutoring systems. In: LAK21, pp. 161–172 (2021)

    Google Scholar 

  10. Jin, T., Lu, X.: A data-driven approach to text adaptation in teaching material preparation: design, implementation, and teacher professional development. TESOL Q. 52(2), 457–467 (2018)

    Article  Google Scholar 

  11. Koedinger, K.R., Aleven, V.: An interview reflection on “intelligent tutoring goes to school in the big city”. IJAIED 26(1), 13–24 (2016)

    Google Scholar 

  12. Koedinger, K.R., Anderson, J.R.: Illustrating principled design: the early evolution of a cognitive tutor for algebra symbolization. Interact. Learn. Environ. 5(1), 161–179 (1998)

    Article  Google Scholar 

  13. Koedinger, K.R., Stamper, J.C., Leber, B., Skogsholm, A.: LearnLab’s datashop: a data repository and analytics tool set for cognitive science. Top. Cogn. Sci. 5(3), 668–669 (2013)

    Article  Google Scholar 

  14. Koedinger, K.R., Stamper, J.C., McLaughlin, E.A., Nixon, T.: Using data-driven discovery of better student models to improve student learning. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 421–430. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_43

    Chapter  Google Scholar 

  15. Long, Y., Aleven, V.: Enhancing learning outcomes through self-regulated learning support with an open learner model. User Model. User-Adap. Inter. 27, 55–88 (2017)

    Article  Google Scholar 

  16. McBroom, J., Yacef, K., Koprinska, I., Curran, J.R.: A data-driven method for helping teachers improve feedback in computer programming automated tutors. In: Penstein Rosé, C., et al. (eds.) AIED 2018. LNCS (LNAI), vol. 10947, pp. 324–337. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93843-1_24

    Chapter  Google Scholar 

  17. Murray, T., Woolf, B.P.: Tools for teacher participation in ITS design. In: Frasson, C., Gauthier, G., McCalla, G.I. (eds.) ITS 1992. LNCS, vol. 608, pp. 593–600. Springer, Heidelberg (1992). https://doi.org/10.1007/3-540-55606-0_69

    Chapter  Google Scholar 

  18. Norman, G.: Likert scales, levels of measurement and the “laws’’ of statistics. Adv. Health Sci. Educ. 15(5), 625–632 (2010)

    Article  Google Scholar 

  19. Simon, H.A., Newell, A.: Human problem solving: the state of the theory in 1970. Am. Psychol. 26(2), 145 (1971)

    Article  Google Scholar 

  20. Tsung, S., Wei, H., Li, H., Wang, Y., Xia, M., Qu, H.: Blocklens: visual analytics of student coding behaviors in block-based programming environments. In: Proceedings of the Ninth ACM Conference on Learning@ Scale, pp. 299–303 (2022)

    Google Scholar 

  21. VanLehn, K.: The behavior of tutoring systems. AIED 16(3), 227–265 (2006)

    Google Scholar 

  22. Xia, M., et al.: Peerlens: peer-inspired interactive learning path planning in online question pool. In: Proceedings of the 2019 CHI, pp. 1–12 (2019)

    Google Scholar 

  23. Xia, M., Velumani, R.P., Wang, Y., Qu, H., Ma, X.: QLens: visual analytics of multi-step problem-solving behaviors for improving question design. IEEE Trans. Visual Comput. Graph. 27(2), 870–880 (2020)

    Article  Google Scholar 

  24. Xia, M., et al.: Visual analytics of student learning behaviors on k-12 mathematics e-learning platforms. arXiv preprint arXiv:1909.04749 (2019)

  25. Yang, K.B., Nagashima, T., Yao, J., Williams, J.J., Holstein, K., Aleven, V.: Can crowds customize instructional materials with minimal expert guidance? Exploring teacher-guided crowdsourcing for improving hints in an ai-based tutor. Proc. ACM Hum.-Comput. Interact. 5(CSCW1), 1–24 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, USA

    Meng Xia, Yun Huang, Jonathan Sewall & Vincent Aleven

  2. Fudan University, Shanghai, China

    Xinyi Zhao

  3. Nio Inc., Shanghai, China

    Dong Sun

Authors
  1. Meng Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinyi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yun Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jonathan Sewall
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vincent Aleven
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng Xia .

Editor information

Editors and Affiliations

  1. University of Southern California, Los Angeles, CA, USA

    Ning Wang

  2. University of British Columbia, Vancouver, BC, Canada

    Genaro Rebolledo-Mendez

  3. North Carolina State University, Raleigh, NC, USA

    Noboru Matsuda

  4. Despacho 3.01, UNED-Grupo de Investigación aDeNu, Madrid, Spain

    Olga C. Santos

  5. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xia, M., Zhao, X., Sun, D., Huang, Y., Sewall, J., Aleven, V. (2023). Involving Teachers in the Data-Driven Improvement of Intelligent Tutors: A Prototyping Study. In: Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O.C., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2023. Lecture Notes in Computer Science(), vol 13916. Springer, Cham. https://doi.org/10.1007/978-3-031-36272-9_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-36272-9_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-36271-2

  • Online ISBN: 978-3-031-36272-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation