Skip to main content
Log in

A generalized multi-skill aggregation method for cognitive diagnosis

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Online education brings more possibilities for personalized learning, in which identifying the cognitive state of learners is conducive to better providing learning services. Cognitive diagnosis is an effective measurement to assess the cognitive state of students through response data of answering the problems(e.g., right or wrong). Generally, the cognitive diagnosis framework includes the mastery of skills required by a specified problem and the aggregation of skills. The current multi-skill aggregation methods are mainly divided into conjunctive and compensatory methods and generally considered that each skill has the same effect on the correct response. However, in practical learning situations, there may be more complex interactions between skills, in which each skill has different weight impacting the final result. To this end, this paper proposes a generalized multi-skill aggregation method based on the Sugeno integral (SI-GAM) and introduces fuzzy measures to characterize the complex interactions between skills. We also provide a new idea for modeling multi-strategy problems. The cognitive diagnosis process is implemented by a more general and interpretable aggregation method. Finally, the feasibility and effectiveness of the model are verified on synthetic and real-world datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. https://github.com/kathy-sj/SI-GAM

References

  1. Anderson, D.T., Keller, J.M., Havens, T.C.: Learning fuzzy-valued fuzzy measures for the fuzzy-valued sugeno fuzzy integral. In: International Conference on Information Processing and Management of UncertaintyIPMU 2010 (2010)

  2. Beliakov, G., Divakov, D.: On representation of fuzzy measures for learning choquet and sugeno integrals. Knowl Based Syst 189(10), 105134 (2019)

    Google Scholar 

  3. bigdata-ustc: EduData. https://github.com/bigdata-ustc/EduData. Accessed 2022-04-22

  4. Bin, Q., Hongxia, Z., Yu, W., Jixing, L.: Questions recommendation based on collaborative filtering and cognitive diagnosis. Computer Science 46(11), 235–240 (2019)

    Google Scholar 

  5. Birnbaum, A.: Some latent trait models and their use in inferring an examinee’s ability (1968)

  6. Carlson, J.E., von Davier, M.: Item response theory. Ets Research Report 2013(2), i–69 (2013)

    Google Scholar 

  7. Chakraborty, M., Biswas, S.K., Purkayastha, B.: Rule extraction from neural network trained using deep belief network and back propagation. Knowl Inf Syst 62(17) (2020)

  8. Cheng, Y.: Computerized adaptive testing—new developments and applications. Dissertations & Theses - Gradworks (2008)

  9. De La Torre, J.: Dina model and parameter estimation: A didactic. J Educ Behav Stat 34(1), 115–130 (2009)

    Article  Google Scholar 

  10. De La Torre, J., Douglas, J.A.: Model evaluation and multiple strategies in cognitive diagnosis: ananalysis of fraction subtraction data. Psychometrika 73(4), 595–624 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  11. De La Torre, J., Lee, Y.S.: A note on the invariance of the dina model parameters. J Educ Meas 47(1), 115–127 (2010)

    Article  Google Scholar 

  12. De La Torre, J., Minchen, N.: Cognitively diagnostic assessments and the cognitive diagnosis model framework. Psicol Educ 20(2), 89–97 (2014)

    Article  Google Scholar 

  13. Delatorre, J.: The generalized dina model framework. Psychometrika 76(2), 179–199 (2011). https://doi.org/10.1007/s11336-011-9207-7

    Article  MathSciNet  MATH  Google Scholar 

  14. DONGbo, T.: A polytomous cognitive diagnosis model: P-dina model. Acta Psychologica Sinica 42(10), 1011–1020 (2010)

  15. Geyer, C.J.: Markov chain monte carlo maximum likelihood. American Cancer Society (2005)

  16. Gierl, M.J.: Making diagnostic inferences about cognitive attributes using the rule-space model and attribute hierarchy method. J Educ Meas 44(4), 325–340 (2007)

    Article  Google Scholar 

  17. Grabisch, M., Murofushi, T., Sugeno, M.: Fuzzy measures and integrals–theory and applications. Medical Tribune (2000)

  18. Grabisch, M.: Set functions, games and capacities in decision making. Université Paris1 Panthéon Sorbonne (2016)

  19. Grabisch, M.: The representation of importance and interaction of features by fuzzy measures. Pattern Recognit Lett 17(6), 567–575 (1996)

    Article  Google Scholar 

  20. Grabisch, M., Labreuche, C.: A decade of application of the choquet and sugeno integrals in multi-criteria decision aid. Ann Oper Res 175(1), 247–286 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  21. Guo, L., Bao, Y., Wang, Z., Bian, Y.: Cognitive diagnostic assessment with different weight for attribute: based on the dina model. Psychol Rep 114(3), 802–822 (2014)

    Article  Google Scholar 

  22. Haertel, E.H.: Using restricted latent class models to map the skill structure of achievement items. J Educ Meas 26(4), 301–321 (1989)

    Article  Google Scholar 

  23. Hartz, S.M.: A bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Ph.D. thesis, ProQuest Information & Learning (2002)

  24. Hastings, W.K.: Monte carlo sampling methods using markov chains and their applications. Biometrika 1, 97–109 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  25. Islam, M.A., Anderson, D.T., Pinar, A.J., Havens, T.C., Scott, G., Keller, J.M.: Enabling explainable fusion in deep learning with fuzzy integral neural networks. IEEE Transactions on Fuzzy Systems (2019)

  26. Item response theory - application to psychological measurement. British Journal of Surgery (1983)

  27. Junker, B.W., Sijtsma, K.: Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Appl Psychol Meas 25(3), 258–272 (2001)

    Article  MathSciNet  Google Scholar 

  28. Kaya, Y., Leite, W.L.: Assessing change in latent skills across time with longitudinal cognitive diagnosis modeling: An evaluation of model performance. Educ Psychol Meas 77(3) (2016)

  29. Leighton, J.P., Gierl, M.J., Hunka, S.M.: The attribute hierarchy method for cognitive assessment: a variation on tatsuoka ë rule-space approach. J Educ Meas 41(3), 205–237 (2004)

  30. Liu, Q., Wu, R., Chen, E., Xu, G., Su, Y., Chen, Z., Hu, G.: Fuzzy cognitive diagnosis for modelling examinee performance. ACM Trans Intell Syst Technol 9(4), 1–26 (2018)

    Article  Google Scholar 

  31. Liu, Q., Huang, Z., Yin, Y., Chen, E., Xiong, H., Su, Y., Hu, G.: Ekt: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering 33(1), 100–115 (2021)

    Article  Google Scholar 

  32. Ma, W., Guo, W.: Cognitive diagnosis models for multiple strategies. Br J Math Stat Psychol (2019)

  33. Ma, W., Guo, W.: Cognitive diagnosis models for multiple strategies. Br J Math Stat Psychol 72(2), 370–392 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  34. Maris, E.: Estimating multiple classification latent class models. Psychometrika 64(2), 187–212 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  35. Murillo, J., Guillaume, S., Bulacio, P.: k-maxitive fuzzy measures: A scalable approach to model interactions. Fuzzy Sets Syst 324, 33–48 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  36. Pardos, Z.A., Heffernan, N.T., Ruiz, C., Beck, J.E.: The composition effect: conjuntive or compensatory? an analysis of multi-skill math questions in its. In: International Conference on Educational Data Mining (2008)

  37. Rasch, G.: Studies in mathematical psychology: I. probabilistic models for some intelligence and attainment tests (1960)

  38. Sugeno, M.: Theory of fuzzy integrals and its applications. Ph.D. thesis, Institute of Technology (1974)

  39. Sun, J., Xin, T., Zhang, S., de la Torre, J.: A polytomous extension of the generalized distance discriminating method. Appl Psychol Meas 37(7), 503–521 (2013)

    Article  Google Scholar 

  40. Tatsuoka, K.K.: Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. In: Presented at the Office of Naval Research Contractors Conference in Iowa City, IA, May 1993. Lawrence Erlbaum Associates, Inc (1995)

  41. Tatsuoka, K.K.: Boolean algebra applied to determination of universal set of knowledge states. ETS Research Report Series 1991(2), i–36 (1991)

    Google Scholar 

  42. Tatsuoka, K.K.: Rule space: An approach for dealing with misconceptions based on item response theory. J Educ Meas 20(4), 345–354 (2005)

    Article  Google Scholar 

  43. Templin, J., Henson, R.: Measurement of psychological disorders using cognitive diagnosis models. Psychol Methods 11(3), 287–305 (2006)

    Article  Google Scholar 

  44. Torra, V., Narukawa, Y.: The interpretation of fuzzy integrals and their application to fuzzy systems. International Journal of Approximate Reasoning 41(1), 43–58 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  45. Torre, J.D.L., Douglas, J.A.: Higher-order latent trait models for cognitive diagnosis. Psychometrika 69(3), 333–353 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  46. Tu, D.B., Cai, Y., DIng, S.L.: A new multiple-strategies cognitive diagnosis model: the mscd method. Acta Psychologica Sinica 44(011), 1547–1553 (2012)

  47. Wang, D., Cai, Y., Tu, D.: Q-matrix estimation methods for cognitive diagnosis models: based on partial known q-matrix. Multivariate Behav Res pp. 1–13 (2020)

  48. Wei, G.W., Wang, H.J., Lin, R.: Application of correlation coefficient to interval-valued intuitionistic fuzzy multiple attribute decision-making with incomplete weight information. Knowl Inf Syst (2011). https://doi.org/10.1007/s10115-009-0276-1

    Article  Google Scholar 

  49. Wu, R., Liu, Q., Liu, Y., Chen, E., Su, Y., Chen, Z., Hu, G.: Cognitive modelling for predicting examinee performance. In: Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI’15, pp. 1017–1024. AAAI Press (2015)

  50. Yue, Kou: Derong, Shen, Hongbin, Xu, Menger, Lin, Ge, Yu: Two-level interactive identification and derivation of topic clusters in complex networks. World Wide Web 18(4), 1093–1122 (2015)

    Article  Google Scholar 

  51. Zhang, M., Zhu, J., Wang, Z., Chen, Y.: Providing personalized learning guidance in moocs by multi-source data analysis. World Wide Web 22, 1189–1219 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

The research was partially supported by National Key R& D Program of China (No: 2018YFB1403400), the National Education Scientific Planning Projects (Grants No. BCA190081).

Funding

The project is supported by National Key R& D Program of China (No: 2018YFB1403400) and National Education scientific Planning Projects (Grants No. BCA190081).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Song Huang.

Ethics declarations

Conflicts of interest

The authors have no financial or proprietary interests in any material discussed in this article.

Appendix

Appendix

The derivation of the latent response under three assumptions:

  • Case 1. If \(\alpha _{(1)^{\prime }}<\alpha _{(1)^{\prime \prime }}\), then

    • If \(\alpha _{(k)}=\alpha _{(1)^{\prime }}\), then \(X_1\subseteq L_k\) and \(X_1\not \subseteq L_{k+1}\), thus \(\mu (L_k)=1\). Since \(\alpha _{(1)^{\prime }}<\alpha _{(1)^{\prime \prime }}\), \(X_2\subseteq L_k\) and \(X_2\subseteq L_{k+1}\). It is easy to show that the result is contrary to the (20), it does not meet the constraints.

    • Else if \(\alpha _{(k)}=\alpha _{(1)^{\prime \prime }}\), then \(X_2\subseteq L_k\), and \(X_2\not \subseteq L_{k+1}\). As \(\alpha _{(1)^{\prime }} < \alpha _{(1)^{\prime \prime }}\), we can infer that \(X_1\not \subseteq L_k\) and \(X_1\not \subseteq L_{k+1}\). Hence, we use \(\alpha _{(k)}=\alpha _{(1)^{\prime \prime }}\) to substitute into the calculation formula, which satisfies the (20), that is \(\mu (L_k)=1\) and \(\mu (L_{k+1})=0\).

    Thus, the result of the latent response is given by \(\eta =\alpha _{(k)}=\alpha _{(1)^{\prime \prime }}\), when \(\alpha _{(1)^{\prime }}<\alpha _{(1)^{\prime \prime }}\).

  • Case 2. If \(\alpha _{(1)^{\prime }}=\alpha _{(1)^{\prime \prime }}\), in other words, \(\alpha _{(k)}=\alpha _{(1)^{\prime }}=\alpha _{(1)^{\prime \prime }}\), then \(X_1\subseteq L_k\), \(X_2\subseteq L_k\), \(X_1\not \subseteq L_{k+1}\) and \(X_2\not \subseteq L_{k+1}\). This inference meets the constraints of (20), thus \(\mu (L_k)=1\) and \(\mu (L_{k+1})=0\) . Moreover, the latent response equals to \(\eta =\alpha _{(k)}=\alpha _{(1)^{\prime }}=\alpha _{(1)^{\prime \prime }}\), when \(\alpha _{(1)^{\prime }}=\alpha _{(1)^{\prime \prime }}\).

  • Case 3. If \(\alpha _{(1)^{\prime }}>\alpha _{(1)^{\prime \prime }}\), then

    • If \(\alpha _{(k)}=\alpha _{(1)^{\prime }}\), then \(X_1\subseteq L_k\), \(X_1\not \subseteq L_{k+1}\). For \(\alpha _{(1)^{\prime }}>\alpha _{(1)^{\prime \prime }}\), it is clear that \(X_2\not \subseteq L_{k}\) and \(X_2\not \subseteq L_{k+1}\). According to (20), we can conclude that \(\mu (L_k)=1\). Moreover, the latent response is \(\eta =\alpha _{(k)}=\alpha _{(1)^{\prime }}\).

    • Else if \(\alpha _{(k)}=\alpha _{(1)^{\prime \prime }}\), then \(X_2\subseteq L_k\), obviously, \(X_2\not \subseteq L_{k+1}\). Since \(\alpha _{(1)^{\prime }}>\alpha _{(1)^{\prime \prime }}\), we can infer that \(X_1\not \subseteq L_k\) and \(X_1\subseteq L_{k+1}\). Hence, it is impossible that make \(\mu (L_k)=1\) and \(\mu (L_{k+1})=0\). Hence, the latent response is computed by \(\eta =\alpha _{(k)}=\alpha _{(1)^{\prime }}\) when \(\alpha _{(1)^{\prime }}>\alpha _{(1)^{\prime \prime }}\).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, S., Huang, S., Yu, X. et al. A generalized multi-skill aggregation method for cognitive diagnosis. World Wide Web 26, 585–614 (2023). https://doi.org/10.1007/s11280-021-00990-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-021-00990-4

Keywords