The control group is a group of consumers, presumably similar to the experimental group except for not being exposed to the experimental variable. Another type of control could have been purchases of similar, but unadvertised, products by the same consumers. When the experiment is limited to one medium sized Midwestern market, the degree to which the results can be generalized is limited to the one market studied; although strong suggestions may be drawn relative to other medium sized mid western cities and even beyond.
This design has definite advantages over the simple before after design. The effects of the uncontrolled variables, history, maturation, pre-testing, and measurement variability, should be the same for the control group as for the experimental group. In this case, these factors appear to have had a negative effect on purchases of 2.5 percent. If it had not been for the experimental variable, presumably the experimental group would have shown a similar drop in purchases during the period.
Selection of test subjects is a further confounding factor may occur in this design. The selection of people for the experimental and control groups was on the basis of whether or not they were exposed to advertising of the products in question. If the 2,441 subjects had been divided into two groups on a random basis and one group exposed to advertising while the other was not, no selection problem exists. In the example given, however, the subjects were separated on the basis of their own verbal reports of whether or not they had been exposed to the advertising in question. Thus, the respondents in each group were self selected almost automatically guaranteeing that those themselves in the experimental group would show a larger purchase rate for the products in question than those in the control group. Individuals who have purchased a product are more apt to be aware of the advertising than those who have not. The cause and effect relationship might be in reverse order to that inferred from the experiment. More generally, however, self selection is almost sure not to give two groups that are comparable.
If the selection of test subjects for both the experimental and control groups is controlled by the experimenter and if the assignment to each group is on a random basis, the degree to which the two groups might differ can be measured statistically. Such measurements will be discussed in the chapters on sampling. A tendency in early experiments was to match experimental and control groups by making them similar on various characteristics, such as age, sex, income, nationality of origin, and so on. Since it is impossible ever to match two individuals let alone two groups on all possible characteristics the matching of the two groups is only a little better than self selection.
Even though history seems to be adequately controlled in the before after with control group design, it is possible to have biases of this type if the two groups are in different geographical locations or are measured at different times.
Mortality is a factor that become a particularly noticeable source of bias in the before after with control group design. This is the loss of some test subjects between the before and after measurements. Such a loss can occur in the simple before after design, but it is usually ignored for the sake of convenience. It is more serious to ignore losses when the mortality rate is different between the experimental and control groups. Even in the latter case, those who drop out are often disregarded but should not be. In experiments where the experiments where the experimental group has to perform certain tasks during a period of time, mortality may become relatively heavy among the experimental group. For example, if the experimental group has to keep a diary of certain activities, many may become disinterested and fail to keep the diary or do so only in part. This who drop out are apt to be different form the others and so cause variation in the after measurement that would not otherwise have occurred.