Abstract
A trend in educational testing is to go beyond unidimensional scoring and provide a more complete profile of skills that have been mastered and those that have not. To achieve this, cognitive diagnosis models have been developed that can be viewed as restricted latent class models. Diagnosis of class membership is the statistical objective of these models. As an alternative to latent class modeling, a nonparametric procedure is introduced that only requires specification of an item-by-attribute association matrix, and classifies according to minimizing a distance measure between observed responses, and the ideal response for a given attribute profile that would be implied by the item-by-attribute association matrix. This procedure requires no statistical parameter estimation, and can be used on a sample size as small as 1. Heuristic arguments are given for why the nonparametric procedure should be effective under various possible cognitive diagnosis models for data generation. Simulation studies compare classification rates with parametric models, and consider a variety of distance measures, data generation models, and the effects of model misspecification. A real data example is provided with an analysis of agreement between the nonparametric method and parametric approaches.
Similar content being viewed by others
References
AYERS, E., NUGENT, R., and DEAN, N. (2008), ”Skill Set Profile Clustering Based on Student Capability Vectors Computed From Online Tutoring Data”, in Proceedings of the 1st International Conference on Education Data Mining, eds. Baker and Beck, pp. 218–225. Montreal.
BARNES, T. (2010), “Novel Derivation and Application of Skill Matrices: The Q-Matrix Method”, in Handbook on Educational Data Mining, Boca Raton FL: CRC Press, pp. 159–172.
CHIU, C., DOUGLAS, J., and LI, X. (2009), “Cluster Analysis for Cognitive Diagnosis: Theory and Applications”, Psychometrika, 74, 633–665.
DE LA TORRE, J., and DOUGLAS, J. (2004), “Higher-order Latent Trait Models for Cognitive Diagnosis”, Psychometrika, 69, 333–353.
HARTZ, S., ROUSSOS, L., HENSON, R., and TEMPLIN, J. (2005), The Fusion Model for Skill Diagnosis: Blending Theory with Practicality, unpublished manuscript.
HENSON, R., TEMPLIN, J., and WILLSE, J. (2009), “Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables”, Psychometrika, 74, 191–210.
JUNKER, B.W., and SIJTSMA, K. (2001), “Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory”, Applied Psychological Measurement, 25, 258–272.
MARIS, E. (1999), “Estimating Multiple Classification Latent Class Models”, Psychometrika, 64, 187–212.
TATSUOKA, K. (1983), “Rule-Space: An Approach for Dealing with Misconceptions Based on Item Response Theory”, Journal of Educational Measurement, 20, 34–38.
TATSUOKA, K. (1985), “A Probabilistic Model for Diagnosing Misconceptions in the Pattern Classification Approach”, Journal of Educational Statistics, 12, 55–73.
TATSUOKA, K. (1990), “Architecture of Knowledge Structures and Cognitive Diagnosis: A Statistical Pattern Classification Approach”, in Cognitively Diagnostic Assessments, eds. P. Nichols, S. Chipman, and R. Brennan, Hillsdale, NJ: Erlbaum, pp. 327–359.
TATSUOKA, K., and TATSUOKA, M. (1987), “Bug Distribution and Pattern Classification”, Psychometrika, 52, 193–206.
TATSUOKA, K., and TATSUOKA, M. (1997), “Computerized Cognitive Diagnostic Adaptive Testing: Effect on Remedial Instruction as Empirical Validation”, Journal of Educational Measurement, 34, 3–20.
TEMPLIN, J.L., and HENSON, R.A. (2006), “Measurement of Psychological Disorders Using Cognitive Diagnosis Models”, Psychological Methods, 11, 287–305.
WILLSE, J., HENSON, R., and TEMPLIN, J. (2007), “Using Sum Scores or IRT in Place of Cognitive Diagnosis Models: Can Existing or More Familiar Models Do the Job?” paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, Illinois.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chiu, CY., Douglas, J. A Nonparametric Approach to Cognitive Diagnosis by Proximity to Ideal Response Patterns. J Classif 30, 225–250 (2013). https://doi.org/10.1007/s00357-013-9132-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-013-9132-9