The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
On Evaluating Curricular Effectiveness: Judging the Quality of K-12 Mathematics Evaluations
pened” frequently do not provide sufficient experimental evidence to permit causal inference about a curriculum’s effectiveness as measured by student achievement, the studies may indicate why that curriculum had the effect it did and it may highlight aspects of implementation or design that were instrumental in producing that effect. Such studies, generally referred to as case studies, can provide useful information along a number of dimensions that emanate from a careful description of the connections among a curriculum’s program theory, its implementation theory, and its actualization in particular settings (Bickman, 1987). The generalizations from a well-designed comparative evaluation may not provide sufficient information to permit decision makers to know whether the experimental treatment (new curriculum) will be appropriate for their particular setting. Case studies may provide additional specificity that is necessary and helpful to practitioners in assessing the probability of successful use in their settings. As written by Easley (1977, p. 6):
Experimentalists feel that they can generalize their findings from an experiment to the population as a whole because they have drawn an adequate random sample from the population about which a hypothesis speaks. Clinical researchers feel that they can generalize from a study of a single case to some other individual cases because they have seen a given phenomenon in one situation in sufficient detail and know its essential workings to be able to recognize it when they encounter it in another situation.
Criteria for Inclusion in Our Study
Forty-five articles, dissertations, and unpublished manuscripts were originally classified as case studies. We considered case studies, ethnographies, descriptive studies, and research studies that inform us about what happens in the implementation of specific curricula, classifying them all as “case studies” for simplicity. To be classified as a case study, the study had to examine curricula implementation of significant parts of the curricula materials (more than one unit) over a significant duration (more than one semester) and had to show evidence of systematic data collection and report on the effectiveness of the materials in the conclusions. For our purposes the study also had to focus on 1 of the 13 mathematics curricula supported by the National Science Foundation (NSF), the University of Chicago School Mathematics Project (UCSMP) curriculum, or one of the five other commercially generated mathematics curricula included in our review.
After the initial categorizing, we refined our criteria for inclusion in our review to stipulate that the case studies must have been published, be a dissertation, or have a draft date of 2000 or later. We assumed that manuscripts with a draft date prior to 2000 were written with the intent to publish. Therefore we decided not to consider them if they remained un-