Skip to main content
Log in

Using Neural Network Analysis to Define Methods of DINA Model Estimation for Small Sample Sizes

Journal of Classification Aims and scope Submit manuscript

Abstract

The DINA model is a commonly used model for obtaining diagnostic information. Like many other Diagnostic Classification Models (DCMs), it can require a large sample size to obtain reliable item and examinee parameter estimation. Neural Network (NN) analysis is a classification method that uses a training dataset for calibration. As a result, if this training dataset is determined theoretically, as was the case in Gierl’s attribute hierarchical method (AHM), the NN analysis does not have any sample size requirements. However, a NN approach does not provide traditional item parameters of a DCM or allow for item responses to influence test calibration. In this paper, the NN approach will be implemented for the DINA model estimation to explore its effectiveness as a classification method beyond its use in AHM. The accuracy of the NN approach across different sample sizes, item quality and Q-matrix complexity is described in the DINA model context. Then, a Markov Chain Monte Carlo (MCMC) estimation algorithm and Joint Maximum Likelihood Estimation is used to extend the NN approach so that item parameters associated with the DINA model are obtained while allowing examinee responses to influence the test calibration. The results derived by the NN, the combination of MCMC and NN (NN MCMC) and the combination of JMLE and NN are compared with that of the well-established Hierarchical MCMC procedure and JMLE with a uniform prior on the attribute profile to illustrate their strength and weakness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • CASTEJÓN LIMAS, M., and ORDIERES MERÉ, J. (2009), “The AMORE Package: A MORE Flexible Neural Network Package” (R package), http://cran.r-project.org/.

  • DE LA TORRE, J. (2008), “An Empirically Based Method of Q-Matrix Validation for the DINA Model: Development and Applications”, Journal of Educational Measurement , 45(4), 343–362.

    Article  Google Scholar 

  • DE LA TORRE, J. (2009), “DINA Model and Parameter Estimation: A Didactic”, Journal of Educational and Behavioral Statistics, 34(1), 115–130.

    Article  Google Scholar 

  • DE LA TORRE, J., and DOUGLAS, J. (2004), “Higher Order Latent Trait Models for Cognitive Diagnosis, Psychometrika, 69(3), 333–353.

    Article  MathSciNet  Google Scholar 

  • GARSON, G. (1998), Neural Networks: An Introductory Guide for Social Scientists, London: SAGE Publication Ltd.

    Google Scholar 

  • GIERL, M. (2007), “Making Diagnostic Inferences about Cognitive Attributes Using the Rule-Space Model and Attribute Hierarchy Method”, Journal of Educational Measurement, 44(4), 325–340.

    Article  Google Scholar 

  • GIERL, M.J., Cui, Y., and HUNKA, S. (2007), “Using Connectionist Models to Evaluate Examinees’ Response Patterns on Tests”, paper presented at the annual meeting of the National Council on Measurement in Education, Chicago IL.

  • GIERL, M., WANG, C., and ZHOU, J. (2008), “Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees’ Cognitive Skills in Algebra on the SAT”, Journal of Technology, Learning, and Assessment, 6(6).

  • GIERL, M., ZHENG, Y., and CUI, Y. (2008), “Using the Attribute Hierarchy Method to Identify and Interpret Cognitive Skills that Produce Group Differences”, Journal of Educational Measurement, 45(1), 65–89.

    Article  Google Scholar 

  • HAERTEL, E. (1989), ``Using Restricted Latent Class Models to Map the Sill Structure of Achievement Items”, Journal of Educational Measurement (26(4), 301–321.

    Article  Google Scholar 

  • HENSON, R., and TEMPLIN, J. (2006), Implications of Q-matrix Misspecification in Cognitive Diagnosis, manuscript submitted for publication.

  • HENSON, R. (2008), “Functions of Estimating Log-Linear Cognitive Diagnostic Model,” Department of Educational Research Methodology, The University of North Carolina at Greensboro, Greensboro, NC.

    Google Scholar 

  • JUNKER, B.W., and SIJTSMA, K. (2001), “Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory”, Applied Psychological Measurement, 25, 258–272.

    Article  MathSciNet  Google Scholar 

  • LEIGHTON, J., GIERL, M., and HUNKA, S. (2004), “The Attribute Hierarchical Method for Cognitive Assessment: A Variation on Tatsuoka’s Rule-Space Approach”, Journal of Educational Measurement, 41(3), 205–237.

    Article  Google Scholar 

  • PATZ, R., and JUNKER, B. (1999), “A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models”, Journal of Educational and Behavioral Statistics, 24(2), 146–178.

    Google Scholar 

  • RUPP, A., TEMPLIN, J., and HENSON, R. (2010), Diagnostic Measurement: Theory, Methods and Applications, New York, NY: The Guildford Press.

    Google Scholar 

  • STERGIOU, C., and SIGANOS, D., “Neural Networks”, http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html.

  • TEMPLIN, J., HENSON, R., TEMPLIN, S., and ROUSSO, L. (2008), “Robustness of Hierarchical Modeling of Skill Association in Cognitive Diagnosis Models”, Applied Psychological Measurement (OnlineFirst).

  • TEMPLIN, J., and HENSON, R. (2006), ``Measurement of Psychological Disorders Using Cognitive Diagnosis Models”, Psychological Methods 11(3), 287–305.

    Article  Google Scholar 

  • WILLSE, J.T. (2010). “Functions for Estimating DINA and DINO Models Using JML or MML”, Department of Educational Research Methodology, The University of North Carolina at Greensboro, Greensboro, NC.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zhan Shu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shu, Z., Henson, R. & Willse, J. Using Neural Network Analysis to Define Methods of DINA Model Estimation for Small Sample Sizes. J Classif 30, 173–194 (2013). https://doi.org/10.1007/s00357-013-9134-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00357-013-9134-7

Keywords

Navigation