Abstract
Diagnostic classification models (DCMs) are restricted latent class models with a set of cross-class equality constraints and additional monotonicity constraints on their item parameters, both of which are needed to ensure the meaning of classes and model parameters. In this paper, we develop an efficient, Gibbs sampling-based Bayesian Markov chain Monte Carlo estimation method for general DCMs with monotonicity constraints. A simulation study was conducted to evaluate parameter recovery of the algorithm which showed accurate estimation of model parameters. Moreover, the proposed algorithm was compared to a previously developed Gibbs sampling algorithm which imposed constraints on only the main effect item parameters of the log-linear cognitive diagnosis model. The newly proposed algorithm showed less bias and faster convergence. An analysis of the 2000 Programme for International Student Assessment reading assessment data using this algorithm was also conducted.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Brooks, S. P., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434–455. https://doi.org/10.2307/1390675.
Brooks, S., Gelman, A., Jones, G. L., & Meng, X.-L. (2011). Handbook of Markov chain Monte Carlo. CRC press.
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017). Stan : a probabilistic programming language. Journal of Statistical Software, 76. https://doi.org/10.18637/jss.v076.i01.
Chen, Y., Culpepper, S. A., Chen, Y., & Douglas, J. (2018). Baysean estimation of the the DINA Q matrix. Psychometrika, 83, 89–108. https://doi.org/10.1007/s11336-017-9579-4.
Chen, Y., Culpepper, S., & Liang, F. (2020). A sparse latent class model for cognitive diagnosis. Psychometrika, 85(1), 121–153. https://doi.org/10.1007/s11336-019-09693-2.
Chen, J., & de la Torre, J. (2014). A procedure for diagnostically modeling extant large-scale assessment data : the case of the programme for international student assessment in reading. Psychology, 5, 1967–1978. https://doi.org/10.4236/psych.2014.518200.
Chiu, C. Y., & Douglas, J. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30, 225–250. https://doi.org/10.1007/s00357-013-9132-9.
Chung, M. (2019). A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model. Journal of Mathematical Psychology, 93, 102275. https://doi.org/10.1016/j.jmp.2019.07.002.
Culpepper, S. A. (2015). Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics, 40, 454–476. https://doi.org/10.3102/1076998615595403.
Culpepper, S. A. (2019). Estimating the cognitive diagnosis Q matrix with expert knowledge: application to the fraction-subtraction dataset. Psychometrika, 84, 333–357. https://doi.org/10.1007/s11336-018-9643-8.
Culpepper, S. A., & Hudson, A. (2018). An improved strategy for Bayesian estimation of the reduced reparameterized unified model. Applied Psychological Measurement, 42, 99–115. https://doi.org/10.1177/0146621617707511.
de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199. https://doi.org/10.1007/S11336-011-9207-7.
de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353. https://doi.org/10.1007/BF02295640.
DeCarlo, L. T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36, 447–468. https://doi.org/10.1177/0146621612449069.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Erlbaum.
Fang, G., Liu, J., & Ying, Z. (2019). On the identifiability of diagnostic classification models. Psychometrika, 84(1), 19–40. https://doi.org/10.1007/s11336-018-09658-x.
Gelman, A., Meng, X.-L., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6(4), 733–760 https://www.jstor.org/stable/24306036.
George, A. C., Robitzsch, A., Kiefer, T., Groß, J., & Ünlü, A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74. https://doi.org/10.18637/jss.v074.i02.
Gu, Y., & Xu, G. (2019). The sufficient and necessary condition for the identifiability and estimability of the DINA model. Psychometrika, 84(2), 468–483. https://doi.org/10.1007/s11336-018-9619-8.
Gu, Y., & Xu, G. (2020). Partial identifiability of restricted latent class models. Annals of Statistics, 48(4), 2082–2107. https://doi.org/10.1214/19-AOS1878.
Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301–321. https://doi.org/10.1111/j.1745-3984.1989.tb00336.x.
Hartz, S., & Roussos, L. (2008). The fusion model for skills diagnosis: Blending theory with practice. ETS Research Report Series, 08–71, 1–57 Retrieved from https://www.ets.org/Media/Research/pdf/RR-08-71.pdf.
Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210. https://doi.org/10.1007/S11336-008.
Hoijtink, H. (1998). Constrained latent class analysis using the Gibbs sampler and posterior predictive P-values: applications to educational testing. Statistica Sinica, 8, 691–711.
Hong, C.-Y., Chang, Y.-W., & Tsai, R.-C. (2016). Estimation of generalized DINA model with order restrictions. Journal of Classification, 33, 460–484. https://doi.org/10.1007/s0035.
Hu, B., & Templin, J. (2019). Using diagnostic classification models to validate attribute hierarchies and evaluate model fit in Bayesian networks. Multivariate Behavioral Research, 55, 300–311. https://doi.org/10.1080/00273171.2019.1632165.
Jiang, Z., & Carter, R. (2018). Using Hamiltonian Monte Carlo to estimate the log-linear cognitive diagnosis model via Stan, (2014).
Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272. https://doi.org/10.1177/01466210122032064.
Laudy, O., Boom, J., & Hoijtink, H. (2004). Bayesian computational methods for inequality constrained latent class analysis. New Developments in Categorical Data Analysis for the Social and Behavioral Sciences, 52–69. https://doi.org/10.4324/9781410612021
Leighton, J. P., & Gierl, M. J. (Eds.). (2007). Cognitive diagnostic assessment for education: theory and applications. Cambridge University Press.
Li, F., Cohen, A., Bottge, B., & Templin, J. (2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76, 181–204. https://doi.org/10.1177/0013164415588946.
Li, H., Hunter, C. V., & Lei, P.-W. (2016). The selection of cognitive diagnostic models for a reading comprehension test. Language Testing, 33, 1–35. https://doi.org/10.1177/0265532215590848.
Liu, X., & Johnson, M. S. (2019). Estimating CDMs using MCMC. In M. von Davier & Y.-S. Lee (Eds.), Handbook of Diagnostic Classification Models (pp. 629–649). Chem. https://doi.org/10.1007/978-3-030-05584-4_31.
Liu, J., Xu, G., & Ying, Z. (2013). Theory of self-learning Q-matrix. Bernoulli, 19(5 A), 1790–1817. https://doi.org/10.3150/12-BEJ430
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS—a Bayesian modelling framework: concepts, structure, and extensibility, 325–337.
Macready, G. B., & Dayton, C. M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 2, 99–120.
Madison, M. J., & Bradshaw, L. P. (2018). Assessing growth in a diagnostic classification model framework. Psychometrika, 83, 963–990. https://doi.org/10.1007/s11336-018-9638-5.
Meng, X.-L. (1994). Posterior predictive p-values. The Annals of Statistics, 22(3), 1142–1160. https://doi.org/10.1214/aos/1176348654.
Muthén, L. K., & Muthén, B. O. (1998-2017). Mplus User’s Guide (8th ed.). Muthén & Muthén.
Papastamoulis, P. (2016). Label.switching: an R package for dealing with the label switching problem in MCMC outputs. Journal of Statistical Software, 69, 1–11. https://doi.org/10.18637/jss.v069.c01.
Park, Y. S., & Lee, Y.-S. (2014). An extension of the DINA model using covariates: examining factors affecting response probability and latent classification. Applied Psychological Measurement, 38, 376–390. https://doi.org/10.1177/0146621614523830.
Plummer, M. (2003). JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling. The 3rd International Workshop on Distributed Statistical Computing, 124, 1–8. Retrieved from http://www.ci.tuwien.ac.at/Conferences/DSC-2003/
Plummer, M. (2017). JAGS Version 4.3.0 user manual. Retrieved from https://people.stat.sc.edu/hansont/stat740/jags_user_manual.pdfPlummer,
Plummer, M., Best, N., Cowles, K., & Vines, K. (2006). CODA: Convergence diagnosis and output analysis for MCMC. R News, 6, 7–11 Retrieved from https://journal.r-project.org/archive/.
R Core Team. (2019). R: a language and environment for statistical computing. R Foundation for Statistical Computing URL: https://www.R-project.org/.
Rupp, A. A., & Templin, J. (2008). Unique characteristics of diagnostic classification models: a comprehensive review of the current state-of-the-art. Measurement: Interdisciplinary Research & Perspective, 6, 219–262. https://doi.org/10.1080/15366360802490866.
Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic measurement: theory, methods and applications. Guilford Press.
Stephens, M. (2000). Dealing with label switching in mixture models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 62(4), 795–809.
Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354. https://doi.org/10.1111/j.1745-3984.1983.tb00212.x
Tatsuoka, K. K., & Tatsuoka, M. M. (1997). Computerized cognitive diagnostic adaptive testing: effect on remedial instruction as empirical validation. Journal of Educational Measurement, 34, 3–20. https://doi.org/10.1111/j.1745-3984.1997.tb00504.x.
Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: a family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317–339. https://doi.org/10.1007/s11336-013-9362-0.
Templin, J. L., & Henson, R. a. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305. https://doi.org/10.1037/1082-989X.11.3.287.
Templin, J., & Hoffman, L. (2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32, 37–50. https://doi.org/10.1111/emip.12010.
von Davier, M. (2008). A general diagnostic model applied to language testing data. The British Journal of Mathematical and Statistical Psychology, 61, 287–307. https://doi.org/10.1348/000711007X193957.
von Davier, M. (2014). The log-linear cognitive diagnostic model (LCDM) as a special case of the general diagnostic model (GDM). ETS Research Report Series (Vol. RR–14-40). Princeton, NJ. https://doi.org/10.1002/ets2.12043
Watanabe, S. (2018). Mathematical theory of Bayesian statistics. Chapman and Hall/CRC.
Xu, G. (2017). Identifiability of restricted latent class models with binary responses. Annals of Statistics, 45(2), 675–707. https://doi.org/10.1214/16-AOS1464.
Xu, G., & Shang, Z. (2018). Identifying latent structures in restricted latent class models. Journal of the American Statistical Association, 113(523), 1284–1295. https://doi.org/10.1080/01621459.2017.1340889.
Xu, G., & Zhang, S. (2016). Identifiability of diagnostic classification models. Psychometrika, 81(3), 625–649. https://doi.org/10.1007/s11336-015-9471-z.
Yamaguchi, K., & Okada, K. (2018). Comparison among cognitive diagnostic models for the TIMSS 2007 fourth grade mathematics assessment. PLoS One, 13, e0188691. https://doi.org/10.1371/journal.pone.0188691.
Yamaguchi, K., & Okada, K. (2021). Variational Bayes inference algorithm for the saturated diagnostic classification model. Psychometrika, 85, 973–995. https://doi.org/10.1007/s11336-020-09739-w.
Zhan, P., Jiao, H., Man, K., & Wang, L. (2019). Using JAGS for Bayesian cognitive diagnosis modeling: a tutorial. Journal of Educational and Behavioral Statistics, 44, 473–503. https://doi.org/10.3102/1076998619826040.
Funding
This work was supported by JSPS Grant-in-Aid for JSPS Research Fellow 18J01312 and JSPS KAKANHI 20H01720. Jonathan Templin was supported by grants from the National Science Foundation (DRL-1813760) and the Institute of Education Sciences (R305A190079).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Data analysis syntax is available in Open Science Framework, page: https://osf.io/h9fm4/.
Rights and permissions
About this article
Cite this article
Yamaguchi, K., Templin, J. A Gibbs Sampling Algorithm with Monotonicity Constraints for Diagnostic Classification Models. J Classif 39, 24–54 (2022). https://doi.org/10.1007/s00357-021-09392-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-021-09392-7