  Diagnostic classification models (DCMs) are psychometric models for
evaluating a student's mastery of the essential skills in a content domain
based upon their responses to a set of test items. Currently, diagnostic model
and/or Q-matrix misspecification is a known problem with limited avenues for
remediation. To address this problem, this paper defines a one-sided score
statistic that is a computationally efficient method for detecting
under-specification of both the Q-matrix and the model parameters of the
particular DCM chosen in the analysis. This method is analogous to the
modification indices widely used in structural equation modeling. The results
of a simulation study show the Type I error rate of modification indices for
DCMs are acceptably close to the nominal significance level when the
appropriate mixture chi-squared reference distribution is used. The simulation
results indicate that modification indices are very powerful in the detection
of an under-specified Q-matrix and have ample power to detect the omission of
model parameters in large samples or when the items are highly discriminating.
An application of modification indices for DCMs to analysis of response data
from a large-scale administration of a diagnostic test demonstrates how they
can be useful in diagnostic model refinement.
