<p>It depends on what you mean by manipulation check.</p>

<p><strong>Manipulation check to filter participants:</strong> A manipulation check is usually done to verify that some experimental manipulation has had some preliminary effect. For example, in one of my studies I asked people to answer a personality test (a) honestly, (b) as an ideal worker, and (c) as the worst possible worker. I asked each participant how they intended to answer the personality test. The <strong>manipulation check</strong> was there to verify that they had understood the instructions. In this case, the manipulation check was used to potentially exclude participants who failed to pass the manipulation check.</p>

<p><strong>Manipulation check to verify group-level effect:</strong> Of course, you could perform such manipulation checks on an entire group using group comparison tools such as ANOVA for a numeric variable or chi-square for a categorical outcome. For example, in some psychological studies, the experimenter may attempt to manipulate mood as baseline manipulation. Thus, they may want to get check whether the group that has been made to experience negative affect does on average experience more negative affect. In that case an ANOVA makes sense.</p>

<p><strong>Checking approximate group equivalence::</strong> A third case is where you are merely checking whether the groups are the same at baseline and that has more to do with how participants were allocated to groups. It doesn't sound like you are talking about this case.</p>

<p><strong>Using manipulation check outcome as covariate:</strong> In general, manipulation checks are typically reported prior to conducting main analyses. In this sense, they are a bit like assumption tests. They verify that some experimental manipulation has worked and that you can now proceed to examine the effects of substantive interest. That said, you could do a range of other analyses where the outcome measure in the manipulation check is included in your analysis of the substantive outcome. For example, if you were studying task performance in two groups, one with a positive mood induction and one with a negative mood induction, you could run an ANCOVA where condition was your independent variable, measured mood (i.e., the manipulation check variable) was your covariate, and task performance was your dependent variable.</p>

CC BY-SA 3.0
2015-08-07T13:18:50.117Z
Awesome answer, thank you very much
z8080
CC BY-SA 3.0
2015-08-06T01:28:50.753Z
2015-08-06T01:28:50.753Z
Jeromy Anglim
<p>How is doing a so-called manipulation check (i.e. checking to see whether the different groups actually differ for different levels of the independent variable) different from doing - as a main analysis - e.g., an ANOVA to see if the independent variable has had an effect on the dependent variable? From reading papers, it seems people just do manipulation checks <em>prior to</em> the main analyses.</p>

<methodology><experimental-psychology><validation>
How is doing a manipulation check different from doing the analysis on the main dependent variable?
z8080
# How is doing a manipulation check different from doing the analysis on the main dependent variable?

(Asked by: username_0 on 2015-08-05)

How is doing a so-called manipulation check (i.e. checking to see whether the different groups actually differ for different levels of the independent variable) different from doing - as a main analysis - e.g., an ANOVA to see if the independent variable has had an effect on the dependent variable? From reading papers, it seems people just do manipulation checks *prior to* the main analyses.

**Tags:** methodology, experimental-psychology, validation


---

## 1 answers

### Answer by username_1 on 2015-08-06. Score: 3 | Accepted answer

It depends on what you mean by manipulation check.

**Manipulation check to filter participants:** A manipulation check is usually done to verify that some experimental manipulation has had some preliminary effect. For example, in one of my studies I asked people to answer a personality test (a) honestly, (b) as an ideal worker, and (c) as the worst possible worker. I asked each participant how they intended to answer the personality test. The **manipulation check** was there to verify that they had understood the instructions. In this case, the manipulation check was used to potentially exclude participants who failed to pass the manipulation check.

**Manipulation check to verify group-level effect:** Of course, you could perform such manipulation checks on an entire group using group comparison tools such as ANOVA for a numeric variable or chi-square for a categorical outcome. For example, in some psychological studies, the experimenter may attempt to manipulate mood as baseline manipulation. Thus, they may want to get check whether the group that has been made to experience negative affect does on average experience more negative affect. In that case an ANOVA makes sense.

**Checking approximate group equivalence::** A third case is where you are merely checking whether the groups are the same at baseline and that has more to do with how participants were allocated to groups. It doesn't sound like you are talking about this case.

**Using manipulation check outcome as covariate:** In general, manipulation checks are typically reported prior to conducting main analyses. In this sense, they are a bit like assumption tests. They verify that some experimental manipulation has worked and that you can now proceed to examine the effects of substantive interest. That said, you could do a range of other analyses where the outcome measure in the manipulation check is included in your analysis of the substantive outcome. For example, if you were studying task performance in two groups, one with a positive mood induction and one with a negative mood induction, you could run an ANCOVA where condition was your independent variable, measured mood (i.e., the manipulation check variable) was your covariate, and task performance was your dependent variable.

> **Comments:**
> **username_0**: Awesome answer, thank you very much  

---

