Skip to main content
SpringerLink
Log in

Stat-Knowlab. Assessment and Learning of Statistics with Competence-based Knowledge Space Theory

  • Article
  • Published:
International Journal of Artificial Intelligence in Education Aims and scope Submit manuscript

Abstract

An intelligent tutoring system for learning basic statistics, called Stat-Knowlab, is presented and analyzed. The algorithms implemented in the system are based on the competence-based knowledge space theory, a mathematical theory developed for the formative assessment of knowledge and learning. The system’s architecture consists of the two assessment and learning modules that interact with each other in a continuous exchange of information about the current knowledge state of a student. This allows the system to personalize the student’s learning, providing only with the learning objects that she is ready to learn. During the browsing of the system, several types of navigation data are recorded. In this work, we analyzed data from two studies that were aimed at examining the learning processes induced by the navigation of the system. The results of both studies highlighted that the system is useful for monitoring the student learning processes during a university course of basic statistics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Albert, D., & Hockemeyer, C. (2002). Applying demand analysis of a set of test problems for developing adaptive courses. In International conference on computers in education, 2002. proceedings. pp 69–70.

  • Anderson, J.R., Corbett, A.T., Koedinger, K.R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The journal of the learning sciences, 4(2), 167–207.

    Google Scholar 

  • Anselmi, P., Robusto, E., & Stefanutti, L. (2012). Uncovering the best skill multimap by constraining the error probabilities of the gain-loss model. Psychometrika, 77(4), 763–781.

    MathSciNet  MATH  Google Scholar 

  • Anselmi, P., Robusto, E., Stefanutti, L., & de Chiusole, D. (2016). An upgrading procedure for adaptive assessment of knowledge. Psychometrika, 81(2), 461–482.

    MathSciNet  MATH  Google Scholar 

  • Anselmi, P., Stefanutti, L., de Chiusole, D., & Robusto, E. (2017). The assessment of knowledge and learning in competence spaces: The gain–loss model for dependent skills. British Journal of Mathematical and Statistical Psychology, 70(3), 457–479.

    MATH  Google Scholar 

  • Barbara, L.G., & William, L.H. (1996). Enhancing statistics education with expert systems: more than an advisory system. Journal of Statistics Education, 4(3), 1–19.

    Google Scholar 

  • Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), 21(1), 5.

    Google Scholar 

  • Chaiklin, S., & et al. (2003). The zone of proximal development in vygotsky’s analysis of learning and instruction. Vygotsky’s educational theory in cultural context, 1, 39–64.

    Google Scholar 

  • Conati, C., Gertner, A., & Vanlehn, K. (2002). Using bayesian networks to manage uncertainty in student modeling. User modeling and user-adapted interaction, 12(4), 371–417.

    MATH  Google Scholar 

  • Conati, C., & VanLehn, K. (1999). Teaching meta-cognitive skills: Implementation and evaluation of a tutoring system to guide self-explanation while learning from examples. In Artificial intelligence in education, pp 297–304.

  • de Chiusole, D., Anselmi, P., Stefanutti, L., & Robusto, E. (2013). The Gain–Loss Model: bias and variance of the parameter estimates. Electronic Notes in Discrete Mathematics, 42, 33–40.

    MATH  Google Scholar 

  • de Chiusole, D., & Stefanutti, L. (2013). Modeling skill dependence in probabilistic competence structures. Electronic Notes in Discrete Mathematics, 42, 41–48.

    MathSciNet  Google Scholar 

  • de Chiusole, D., Stefanutti, L., Anselmi, P., & Robusto, E. (2015). Modeling missing data in knowledge space theory. Psychological Methods, 20(4), 506–522.

    MATH  Google Scholar 

  • de Chiusole, D., Spoto, A., & Stefanutti, L. (2019). Extracting partially ordered clusters from ordinal polytomous data. Behavior research methods, 52, 503–520.

    Google Scholar 

  • de Chiusole, D., Stefanutti, L., Anselmi, P., & Robusto, E. (2013). Assessing parameter invariance in the blim: Bipartition models. Psychometrika, 78(4), 710–724.

    MathSciNet  MATH  Google Scholar 

  • de Chiusole, D., Stefanutti, L., & Spoto, A. (2017). A class of k-modes algorithms for extracting knowledge structures from data. Behavior research methods, 49(4), 1212–1226.

    Google Scholar 

  • Deonovic, B., Chopade, P., Yudelson, M., de la Torre, J., & von Davier, A.A. (2019). Application of cognitive diagnostic models to learning and assessment systems. In Handbook of diagnostic classification models, pp 437–460. Springer.

  • Doignon, J.-P. (1994). Knowledge spaces and skill assignments. In Fischer, GH, & Laming, D (Eds.) Contributions to mathematical psychology, psychometrics and methodology, pp 111–121. New York: Springer-Verlag.

  • Doignon, J.-P., & Falmagne, J.-C. (1985). Spaces for the assessment of knowledge. International Journal of Man-Machine Studies, 23, 175–196.

    MATH  Google Scholar 

  • Doignon, J.-P., & Falmagne, J.-C. (1999). Knowledge spaces. New York: Springer.

    MATH  Google Scholar 

  • Doignon, J.-P., & Falmagne, J.-C. (1997). Well-graded families of relations. Discrete mathematics, 173(1-3), 35–44.

    MathSciNet  MATH  Google Scholar 

  • Dowling, C.E. (1993). On the irredundant generation of knowledge spaces. Journal of Mathematical Psychology, 37(1), 49–62.

    MathSciNet  MATH  Google Scholar 

  • Düntsch, I, & Gediga, G. (1995). Skills and knowledge structures. British Journal of Mathematical and Statistical Psychology, 48, 9–27.

    MATH  Google Scholar 

  • Falmagne, J.-C., & Doignon, J.-P. (1988). A class of stochastic procedures for the assessment of knowledge. British Journal of Mathematical and Statistical Psychology, 41, 1–23.

    MathSciNet  MATH  Google Scholar 

  • Falmagne, J.-C., & Doignon, J.-P. (1988). A Markovian procedure for assessing the state of a system. Journal of Mathematical Psychology, 32, 232–258.

    MathSciNet  MATH  Google Scholar 

  • Falmagne, J.-C., & Doignon, J.-P. (2011). Learning spaces. New York: Springer.

    MATH  Google Scholar 

  • Falmagne, J.-C., Koppen, M., Villano, M., Doignon, J.-P., & Johanessen, L. (1990). Introduction to knowledge spaces: how to build, test and search them. Psychological Review, 97, 204–224.

    Google Scholar 

  • Falmagne, J.-C., Albert, D., Doble, C., Eppstein, D., & Hu, X. (2013). Knowledge spaces: Applications in education. Berlin: Springer Science & Business Media.

    MATH  Google Scholar 

  • Fang, Y., Ren, Z., Hu, X., & Graesser, A.C. (2019). A meta-analysis of the effectiveness of aleks on learning. Educational Psychology, 39(10), 1278–1292.

    Google Scholar 

  • Gal, I., & Ginsburg, L. (1994). The role of beliefs and attitudes in learning statistics: Towards an assessment framework. Journal of Statistics Education, 2(2), null.

    Google Scholar 

  • Gamboa, H., & Fred, A. (2002). Designing intelligent tutoring systems: A bayesian approach. Enterprise Information Systems III. Edited by J. Filipe, B. Sharp, and P. Miranda. Springer Verlag: New York, pp 146–152.

  • Gediga, G., & Düntsch, I. (2002). Skill set analysis in knowledge structures. British Journal of Mathematical and Statistical Psychology, 55, 361–384.

    MathSciNet  Google Scholar 

  • Haertel, E. (1984). Detection of a skill dichotomy using standardized achievement test items. Journal of Educational Measurement, 21(1), 59–72.

    Google Scholar 

  • Haertel, E.H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26(4), 301–321.

    Google Scholar 

  • Haertel, E.H. (1990). Continuous and discrete latent structure models for item response data. Psychometrika, 55(3), 477–494.

    Google Scholar 

  • Heller, J., Ünlü, A, & Albert, D. (2013). Skills, competencies and knowledge structures. In Falmagne, J.-C., Albert, D., Doble, C., Eppstein, D., & Hu, X. (Eds.) Knowledge spaces: Applications in education, pp 229–242. New York: Springer-Verlag.

  • Heller, J., & Wickelmaier, F. (2013). Minimum discrepancy estimation in probabilistic knowledge structures. Electronic Notes in Discrete Mathematics, 42(4), 49–56.

    MathSciNet  Google Scholar 

  • Heller, J., Anselmi, P., Stefanutti, L., & Robusto, E. (2017). A necessary and sufficient condition for unique skill assessment. Journal of Mathematical Psychology, 79, 23–28.

    MathSciNet  MATH  Google Scholar 

  • Heller, J. (2017). Identifiability in probabilistic knowledge structures. Journal of Mathematical Psychology, 77, 46–57.

    MathSciNet  MATH  Google Scholar 

  • Heller, J., Hockemeyer, C., & Albert, D. (2004). Applying competence structures for peer tutor recommendations in cscl environments. In IEEE International conference on advanced learning technologies, 2004. proceedings. pp 1050–1051.

  • Heller, J., & Repitsch, C. (2012). Exploiting prior information in stochastic knowledge assessment. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 8(1), 12–22.

    Google Scholar 

  • Heller, J., Stefanutti, L., Anselmi, P., & Robusto, E. (2015). On the link between cognitive diagnostic models and knowledge space theory. Psychometrika, 80(4), 995–1019.

    MathSciNet  MATH  Google Scholar 

  • Heller, J., Stefanutti, L., Anselmi, P., & Robusto, E. (2016). Erratum to: On the link between cognitive diagnostic models and knowledge space theory. Psychometrika, 81(1), 250–251.

    MathSciNet  MATH  Google Scholar 

  • Hockemeyer, C., & Albert, D. (1999). The adaptive tutoring system rath. In ICL99 Workshop interactive computer aided learning: Tools and applications. Villach, Austria: Carinthia Tech Institute.

  • Hockemeyer, C., Held, T., & Albert, D. (1997). Rath-a relational adaptive tutoring hypertext www-environment based on knowledge space theory.

  • Kambouri, M., Koppen, M., Villano, M., & Falmagne, J.-C. (1994). Knowledge assessment: Tapping human expertise by the query routine. International Journal of Human-Computer Studies, 40(1), 119–151.

    Google Scholar 

  • Koppen, M. (1993). Extracting human expertise for constructing knowledge spaces: An algorithm. Journal of mathematical psychology, 37(1), 1–20.

    MathSciNet  MATH  Google Scholar 

  • Koppen, M., & Doignon, J.-P. (1990). How to build a knowledge space by querying an expert. Journal of Mathematical Psychology, 34(3), 311–331.

    MathSciNet  MATH  Google Scholar 

  • Korossy, K. (1993). Modellierung von wissen als kompetenz und performance. Eine erweiterung der wissensstruktur-theorie von doignon und falmagne. Ph.D. Thesis. University of Heidelberg.

  • Korossy, K. (1997). Extending the theory of knowledge spaces: A competence-performance approach. Zeitschrift für Psychologie, 205, 53–82.

    Google Scholar 

  • Korossy, K. (1999). Modeling knowledge as competence and performance. In Albert, D., & Lukas, J. (Eds.) Knowledge spaces: Theories, empirical research, applications pp 103–132. Mahwah, NJ: Lawrence Erlbaum Associates.

  • Ma, W., Adesope, O.O., Nesbit, J.C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of educational psychology, 106(4), 901.

    Google Scholar 

  • Mislevy, R.J., & Gitomer, D.H. (1995). The role of probability-based inference in an intelligent tutoring system. ETS Research Report Series, 1995(2), i–27.

    Google Scholar 

  • Mohamed, H., Bensebaa, T., & Trigano, P. (2012). Developing adaptive intelligent tutoring system based on item response theory and metrics. International Journal of Advanced Science and Technology, 43, 1–14.

    Google Scholar 

  • Ohlsson, S. (1994). Constraint-based student modeling. In Student modelling: the key to individualized knowledge-based instruction, pp 167–189. Springer.

  • Psotka, J., Massey, L.D., & Mutter, S.A. (1988). Intelligent tutoring systems: Lessons learned. East Sussex: Psychology Press.

    Google Scholar 

  • Robusto, E., & Stefanutti, L. (2014). Extracting a knowledge structure from the data by a maximum residuals method. TPM: Testing, Psychometrics, Methodology in Applied Psychology.

  • Robusto, E., Stefanutti, L., & Anselmi, P. (2010). The gain-loss model: A probabilistic skill multimap model for assessing learning processes. Journal of Educational Measurement, 47(3), 373–394.

    Google Scholar 

  • Roll, I., Baker, R.S., Aleven, V., & Koedinger, K.R. (2004). A metacognitive act-r model of students’ learning strategies in intelligent tutoring systems. In International Conference on Intelligent Tutoring Systems. pp 854–856.

  • Sargin, A., & Ünlü, A. (2009). Inductive item tree analysis: Corrections, improvements, and comparisons. Mathematical Social Sciences, 58(3), 376–392.

    MathSciNet  MATH  Google Scholar 

  • Schrepp, M. (1999). On the empirical construction of implications between bi-valued test items. Mathematical social sciences, 38(3), 361–375.

    MATH  Google Scholar 

  • Schrepp, M. (2003). A method for the analysis of hierarchical dependencies between items of a questionnaire. Methods of Psychological Research Online, 19, 43–79.

    Google Scholar 

  • Schrepp, M., & Held, T. (1995). A simulation study concerning the effect of errors on the establishment of knowledge spaces by querying experts. Journal of Mathematical Psychology, 39(4), 376–382.

    MATH  Google Scholar 

  • Simon, H.A., & Newell, A. (1971). Human problem solving: The state of the theory in 1970. American Psychologist, 26(2), 145.

    Google Scholar 

  • Spoto, A., Stefanutti, L., & Vidotto, G. (2012). On the unidentifiability of a certain class of skill multi map based probabilistic knowledge structures. Journal of Mathematical Psychology, 56(4), 248–255.

    MathSciNet  MATH  Google Scholar 

  • Spoto, A., Stefanutti, L., & Vidotto, G. (2013). Considerations about the identification of forward-and backward-graded knowledge structures. Journal of Mathematical Psychology, 57(5), 249–254.

    MathSciNet  MATH  Google Scholar 

  • Spoto, A., Stefanutti, L., & Vidotto, G. (2016). An iterative procedure for extracting skill maps from data. Behavior research methods, 48(2), 729–741.

    Google Scholar 

  • Steenbergen-Hu, S., & Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on k–12 students’ mathematical learning. Journal of Educational Psychology, 105(4), 970.

    Google Scholar 

  • Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. Journal of Educational Psychology, 106(2), 331.

    Google Scholar 

  • Stefanutti, L., & Robusto, E. (2009). Recovering a probabilistic knowledge structure by constraining its parameter space. Psychometrika, 74, 83–96.

    MathSciNet  MATH  Google Scholar 

  • Stefanutti, L. (2019). On the assessment of procedural knowledge: From problem spaces to knowledge spaces. British Journal of Mathematical and Statistical Psychology, 72(2), 185–218.

    MATH  Google Scholar 

  • Stefanutti, L., Anselmi, P., & Robusto, E. (2011). Assessing learning processes with the gain-loss model. Behavior research methods, 43(1), 66–76.

    Google Scholar 

  • Stefanutti, L., & de Chiusole, D. (2017). On the assessment of learning in competence based knowledge space theory. Journal of Mathematical Psychology, 80, 22–32.

    MathSciNet  MATH  Google Scholar 

  • Stefanutti, L., Spoto, A., & Vidotto, G. (2018). Detecting and explaining blim’s unidentifiability: Forward and backward parameter transformation groups. Journal of Mathematical Psychology, 82, 38–51.

    MathSciNet  MATH  Google Scholar 

  • Suraweera, P., & Mitrovic, A. (2002). Kermit: A constraint-based tutor for database modeling. In International Conference on Intelligent Tutoring Systems, Springer (pp. 377–387).

  • Symanzik, J., & Vukasinovic, N. (2003). Teaching experiences with a course on “web-based statistics”. The American Statistician, 57(1), 46–50.

    MathSciNet  Google Scholar 

  • Symanzik, J., & Vukasinovic, N. (2006). Teaching an introductory statistics course with cyberstats, an electronic textbook. Journal of Statistics Education, 14(1), 1–9.

    Google Scholar 

  • Templin, J.L., & Henson, R.A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological methods, 11(3), 287.

    Google Scholar 

  • VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.

    Google Scholar 

  • Vanlehn, K., Lynch, C., Schulze, K., Shapiro, J.A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., & Wintersgill, M. (2005). The andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence in Education, 15(3), 147–204.

    MATH  Google Scholar 

  • Xu, Y.J., Meyer, K.A., & Morgan, D. (2008). Piloting a blended approach to teaching statistics in a college of education: Lessons learned. Journal of Educators Online, 5(2), 1–20.

    Google Scholar 

  • Xu, Y.J., Meyer, K.A., & Morgan, D.D. (2009). A mixed-methods assessment of using an online commercial tutoring system to teach introductory statistics. Journal of Statistics Education, 17(2). https://doi.org/10.1080/10691898.2009.11889524.

Download references

Author information

Authors and Affiliations

  1. Department of Philosophy, Sociology, Pedagogy, and Applied Psychology, University of Padua, Via Venezia, 14, 35131, Padova, Italy

    Debora de Chiusole, Luca Stefanutti, Pasquale Anselmi & Egidio Robusto

Authors
  1. Debora de Chiusole
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Stefanutti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pasquale Anselmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Egidio Robusto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Debora de Chiusole.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Chiusole, D., Stefanutti, L., Anselmi, P. et al. Stat-Knowlab. Assessment and Learning of Statistics with Competence-based Knowledge Space Theory. Int J Artif Intell Educ 30, 668–700 (2020). https://doi.org/10.1007/s40593-020-00223-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40593-020-00223-1

Keywords

Search

Navigation