  The increasing availability of longitudinal student achievement data has
heightened interest among researchers, educators and policy makers in using
these data to evaluate educational inputs, as well as for school and possibly
teacher accountability. Researchers have developed elaborate "value-added
models" of these longitudinal data to estimate the effects of educational
inputs (e.g., teachers or schools) on student achievement while using prior
achievement to adjust for nonrandom assignment of students to schools and
classes. A challenge to such modeling efforts is the extensive numbers of
students with incomplete records and the tendency for those students to be
lower achieving. These conditions create the potential for results to be
sensitive to violations of the assumption that data are missing at random,
which is commonly used when estimating model parameters. The current study
extends recent value-added modeling approaches for longitudinal student
achievement data Lockwood et al. [J. Educ. Behav. Statist. 32 (2007) 125--150]
to allow data to be missing not at random via random effects selection and
pattern mixture models, and applies those methods to data from a large urban
school district to estimate effects of elementary school mathematics teachers.
We find that allowing the data to be missing not at random has little impact on
estimated teacher effects. The robustness of estimated teacher effects to the
missing data assumptions appears to result from both the relatively small
impact of model specification on estimated student effects compared with the
large variability in teacher effects and the downweighting of scores from
students with incomplete data.
