<p>You only control for factors that have been proven to matter. IQ has not been shown to be significant. So why measure or control IQ variation if there is no evidence of difference.</p>
<p>In fact there is evidence that the activation of stereotype threat confounds cognitive performance and therefore IQ. If a person has their stereotype threat activated, and they perform poorly in an IQ test, is that a fair representation of their cognitive abilities? Many articles and studies prove the confounding ability of stereotype threat and the need to minimise this factor in IQ testing, it seems complex and unfair to apply a measure that is significantly influenced by the stereotype threat factor that you are studying.</p>
<p><a href="https://spsp.org/news-center/blog/aronson-stereotype-threat" rel="nofollow noreferrer">- Stereotypes impair intellectual performance blog</a></p>

CC BY-SA 4.0
2022-08-02T16:29:19.227Z
Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/138224/discussion-on-answer-by-poidah-why-are-most-psychological-studies-not-controllin).
Arnon Weinberg
CC BY-SA 4.0
2020-11-18T13:05:00.497Z
2020-11-20T08:07:10.503Z
2020-11-20T08:07:10.503Z
Poidah
<p>A quick look at your reference focussed on a meta-analysis of an intervention. Evaluating interventions is mostly about estimating the mean effect of an intervention. So, even if personality or intelligence influences the outcome, this is not directly relevant to evaluating whether on average the intervention has an effect.</p>
<p>In other contexts, controlling for personality and intelligence maybe quite relevant. In general, controlling for personality or intelligence is more relevant (a) where the predictors are conceptually or empirically related to personality or intelligence, and (b) where personality or intelligence is a strong predictor of the outcome of interest.</p>
<p>There are a whole range of psychological constructs that overlap substantially with either personality or intelligence. And in those cases, you do have a body of literature that examines the degree of overlap with personality or intelligence. And there's also literature that examines the incremental prediction of the variable of interest relative to personality.</p>
<p>For example:</p>
<ul>
<li>Grit: Is it different from conscientiousness?</li>
<li>Assessment centers: Do they incrementally predict over and above intelligence?</li>
<li>Type D personality: Is it more than a composite of neurotcism and extraversion</li>
<li>Trait emotional intelligence: Is it just a weighted composite of the Big 5</li>
<li>Ability emotional intelligence: does it predict workplace outcomes over and above intelligence?</li>
</ul>
<p>Researchers are always confronted with issues of which variables to measures. It is often not practical to measure everything. In particular, intelligence takes quite a bit of time to measure well. So you often get research that doesn't control for these factors. And then later on, other studies come along and fill the gap. There is also a bit of a process &quot;construct proliferation&quot; in psychology. A researcher comes up with a new pet construct. And other researchers point out later on how it's not that different to personality or how it doesn't incrementally predict. A lot of heated debates arise over these things.</p>
<p>Researchers also differ in theoretical orientation and practical concerns. For instance, in industrial/organisational psychology, intelligence and Big 5 personality assessment are very well embedded. So, incremental prediction over and above these factors  is a common question for a researchers to ask. In other fields, they may be less grounded in personality or intelligence frameworks.</p>

CC BY-SA 4.0
2020-11-18T01:47:30.307Z
Ideally, in a randomized experiment there should be no differences in your groups in other factors - that's why you randomize. You could still investigate variables that affect the strength of the intervention, but to evaluate the intervention in general it's not necessary. Not sure if the interventions OP referred to involved randomized experiments.
Bryan Krause
CC BY-SA 4.0
2020-11-18T02:27:20.457Z
Hi Bryan. I agree with all your points.
Jeromy Anglim
CC BY-SA 4.0
2020-11-21T11:30:52.327Z
@BryanKrause this is the point I'm struggling with, as a student in statistics, I expect some difference between the groups even if they are sampled from the same population, just because of randomness so I wonder if controlling for iq between the measuring and the control group might not improve our interpretations of the psychological litterature.
outofthegreen
CC BY-SA 4.0
2020-11-21T13:11:22.577Z
@JeandeLéry What about all the other possible confounders you fail to measure? In a small trial, you may have imbalances that affect the result, but not enough data to effectively correct for it. In a larger trial it becomes exceedingly unlikely to find large imbalances on any one variable. Same for meta analyses. Unless there is a bias where a confounder makes people drop out of one arm but not the other, randomization solves the issue.
Bryan Krause
CC BY-SA 4.0
2020-11-21T14:32:11.910Z
I'm no psychology graduate, let alone expert so I thought that IQ and big five were standards like weight and height in biometrics and thus it would make sense to control for these two as when I am doing a linear model in biology I always control for weight & height. But then, if actually there are much more significant cofounders of a size comparable to big 5 and IQ, then my idea was flawed and I am glad you corrected it
outofthegreen
CC BY-SA 4.0
2020-11-18T01:40:39.827Z
2020-11-18T01:40:39.827Z
Jeromy Anglim
<p>Consider for exemple the stereotype threat, of which a recent meta-analysis can be found <a href="https://www.researchgate.net/publication/343149798_Effectiveness_of_Stereotype_Threat_Interventions_A_Meta-Analytic_Review" rel="nofollow noreferrer">here</a>. People try to assess how can a stereotype might explain a difference between two groups.</p>
<p>However, they almost never control for IQ or Big Five which seems to me to be latent variables with much more explanatory power.</p>
<p>My questions are :</p>
<ol>
<li>Why don't they ever control for these two important factors ?</li>
<li>Does it mean that the conclusion of these studies are flawed in some way ?</li>
</ol>

CC BY-SA 4.0
2020-11-17T20:26:04.433Z
Genetics are another major explanatory factor for many psychological constructs, including IQ and personality.  Just think practically what it would take in terms of [power](https://en.wikipedia.org/wiki/Power_of_a_test), to control for such factors - studies would need orders of magnitude more participants.
Arnon Weinberg
CC BY-SA 4.0
2020-11-18T13:35:14.837Z
Yes I understand that it would greatly reduce the power of  the studies but wouldn't it be better to accept null hypothesis because of a too small sample rather than derive conclusions from flawed premisses ?
outofthegreen
CC BY-SA 4.0
2021-08-23T11:51:15.007Z
I always wondered about the opposite. Why would researchers control for all sorts of variables although their theories and hypotheses do not say anything about these variables. Including variables without logic has little to do with the scientific method.
Johannes Titz
<cognitive-psychology><social-psychology><measurement><iq><big-five>
Why are most psychological studies not controlling for IQ and personality scores of their respondents?
outofthegreen
# Why are most psychological studies not controlling for IQ and personality scores of their respondents?

(Asked by: username_0 on 2020-11-17)

Consider for exemple the stereotype threat, of which a recent meta-analysis can be found [here](https://www.researchgate.net/publication/343149798_Effectiveness_of_Stereotype_Threat_Interventions_A_Meta-Analytic_Review). People try to assess how can a stereotype might explain a difference between two groups.

However, they almost never control for IQ or Big Five which seems to me to be latent variables with much more explanatory power.

My questions are :

1. Why don't they ever control for these two important factors ?
2. Does it mean that the conclusion of these studies are flawed in some way ?

**Tags:** cognitive-psychology, social-psychology, measurement, iq, big-five

> **Comments:**
> **username_1**: Genetics are another major explanatory factor for many psychological constructs, including IQ and personality.  Just think practically what it would take in terms of [power](https://en.wikipedia.org/wiki/Power_of_a_test), to control for such factors - studies would need orders of magnitude more participants.  
> **username_0**: Yes I understand that it would greatly reduce the power of  the studies but wouldn't it be better to accept null hypothesis because of a too small sample rather than derive conclusions from flawed premisses ?  
> **username_2**: I always wondered about the opposite. Why would researchers control for all sorts of variables although their theories and hypotheses do not say anything about these variables. Including variables without logic has little to do with the scientific method.  

---

## 2 answers

### Answer by username_4 on 2020-11-18. Score: 4 | Accepted answer

A quick look at your reference focussed on a meta-analysis of an intervention. Evaluating interventions is mostly about estimating the mean effect of an intervention. So, even if personality or intelligence influences the outcome, this is not directly relevant to evaluating whether on average the intervention has an effect.

In other contexts, controlling for personality and intelligence maybe quite relevant. In general, controlling for personality or intelligence is more relevant (a) where the predictors are conceptually or empirically related to personality or intelligence, and (b) where personality or intelligence is a strong predictor of the outcome of interest.

There are a whole range of psychological constructs that overlap substantially with either personality or intelligence. And in those cases, you do have a body of literature that examines the degree of overlap with personality or intelligence. And there's also literature that examines the incremental prediction of the variable of interest relative to personality.

For example:

* Grit: Is it different from conscientiousness?
* Assessment centers: Do they incrementally predict over and above intelligence?
* Type D personality: Is it more than a composite of neurotcism and extraversion
* Trait emotional intelligence: Is it just a weighted composite of the Big 5
* Ability emotional intelligence: does it predict workplace outcomes over and above intelligence?

Researchers are always confronted with issues of which variables to measures. It is often not practical to measure everything. In particular, intelligence takes quite a bit of time to measure well. So you often get research that doesn't control for these factors. And then later on, other studies come along and fill the gap. There is also a bit of a process "construct proliferation" in psychology. A researcher comes up with a new pet construct. And other researchers point out later on how it's not that different to personality or how it doesn't incrementally predict. A lot of heated debates arise over these things.

Researchers also differ in theoretical orientation and practical concerns. For instance, in industrial/organisational psychology, intelligence and Big 5 personality assessment are very well embedded. So, incremental prediction over and above these factors is a common question for a researchers to ask. In other fields, they may be less grounded in personality or intelligence frameworks.

> **Comments:**
> **username_5**: Ideally, in a randomized experiment there should be no differences in your groups in other factors - that's why you randomize. You could still investigate variables that affect the strength of the intervention, but to evaluate the intervention in general it's not necessary. Not sure if the interventions OP referred to involved randomized experiments.  
> **username_4**: Hi Bryan. I agree with all your points.  
> **username_0**: @BryanKrause this is the point I'm struggling with, as a student in statistics, I expect some difference between the groups even if they are sampled from the same population, just because of randomness so I wonder if controlling for iq between the measuring and the control group might not improve our interpretations of the psychological litterature.  
> **username_5**: @JeandeLéry What about all the other possible confounders you fail to measure? In a small trial, you may have imbalances that affect the result, but not enough data to effectively correct for it. In a larger trial it becomes exceedingly unlikely to find large imbalances on any one variable. Same for meta analyses. Unless there is a bias where a confounder makes people drop out of one arm but not the other, randomization solves the issue.  
> **username_0**: I'm no psychology graduate, let alone expert so I thought that IQ and big five were standards like weight and height in biometrics and thus it would make sense to control for these two as when I am doing a linear model in biology I always control for weight & height. But then, if actually there are much more significant cofounders of a size comparable to big 5 and IQ, then my idea was flawed and I am glad you corrected it  

---

### Answer by username_3 on 2020-11-18. Score: 0

You only control for factors that have been proven to matter. IQ has not been shown to be significant. So why measure or control IQ variation if there is no evidence of difference.

In fact there is evidence that the activation of stereotype threat confounds cognitive performance and therefore IQ. If a person has their stereotype threat activated, and they perform poorly in an IQ test, is that a fair representation of their cognitive abilities? Many articles and studies prove the confounding ability of stereotype threat and the need to minimise this factor in IQ testing, it seems complex and unfair to apply a measure that is significantly influenced by the stereotype threat factor that you are studying.

[- Stereotypes impair intellectual performance blog](https://spsp.org/news-center/blog/aronson-stereotype-threat)

> **Comments:**
> **username_1**: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/138224/discussion-on-answer-by-poidah-why-are-most-psychological-studies-not-controllin).  

---

