<p><a href="https://en.wikipedia.org/wiki/Meta-analysis" rel="nofollow noreferrer">Meta-analysis</a> is the statistical practice of combining results across studies in a principled way. Meta-analyses weight the relative strength of evidence from different studies to generate an overall conclusion.</p>
<p>However, there is a bit of art to it, as well, as it's rare that studies are truly fully compatible with one another. They may be in different study populations (for example, sampling from different countries or using different inclusion/exclusion criteria), use different outcome measures, etc. Generally, individual studies also talk about their limitations, for example in the second paper you reference, the authors write about their sample:</p>
<blockquote>
<p>The sample was recruited for a treatment trial for anxiety disorders... Compared to a ... representative sample ... the average score on Internalizing problems was significantly higher for the
children in the current study. ...recruitment strategies might also have caused the
number of children with separation anxiety disorder, social phobia and generalized anxiety disorder to be especially high in our
sample...</p>
</blockquote>
<p>And about their outcome measure:</p>
<blockquote>
<p>In past research, overprotective parenting has been assessed
using different methods and overprotection is only one of several
parenting dimensions that have been considered. The results of the
current study need to be replicated using different measurements
of overprotective parenting and include other parenting dimensions.</p>
</blockquote>
<p>Papers presenting meta analyses often include a qualitative review of the literature, as well, and others may also write <a href="https://en.wikipedia.org/wiki/Review_article" rel="nofollow noreferrer">review articles</a> that summarize a research field without doing a formal meta analysis. These papers will consider various explanations for differences in study results, which can include random statistical variation as well as systematic methodological differences. For example, in the second article you quote from, emphasis mine:</p>
<blockquote>
<p>Results revealed that significant correlations between overprotective parenting and child anxiety symptoms <strong>disappear after controlling for co-occurring child behavior symptoms</strong></p>
</blockquote>
<p>The authors here are describing a departure in their research from what others have done: they've adjusted for &quot;child behavior symptoms&quot;. However, I think they are not really concluding what you think: it isn't that they are saying that there is no link between overprotective parenting and child anxiety, they are saying that overprotective parenting is associated with a wider range of behavioral problems than just anxiety; anxiety isn't unique or special. Because these are all correlational studies (and it isn't really ethical or practically possible to run a randomized experiment where children are assigned to &quot;overprotective&quot; or &quot;not-overprotective&quot; parenting), it's <a href="https://xkcd.com/552/" rel="nofollow noreferrer">not really possible to establish causality</a>, so the causality could be in the opposite direction: that children with behavior problems lead their parents to be overprotective.</p>
<p>If you're looking at a research question that has been studied in multiple separate research papers, I would recommend looking for a recent meta analysis or review paper for further guidance in interpreting the literature.</p>

CC BY-SA 4.0
2022-04-29T23:39:55.210Z
Thanks a lot! I found a meta analysis from 2007, so probably a bit too far back but these types of analysis are definitely something I'll use from now on. What threw me was "child anxiety symptoms disappear after controlling for co-occurring child behavior symptoms" because I interpreted it as saying that overprotection wasn't responsible for anxiety but rather co-occuring behaviours were, and when they were factored out, then there was no relationship between overprotection and anxiety.
Jim stoke
CC BY-SA 4.0
2022-04-30T07:21:00.690Z
Just to follow up, I found a good meta-analysis from this year about the relationship of overprotective parents to internalising/externalising, which is a huge predictor of anxiety. Thanks a lot again! https://onlinelibrary.wiley.com/doi/full/10.1111/sode.12590
Jim stoke
CC BY-SA 4.0
2022-04-29T19:11:38.987Z
2022-04-29T19:11:38.987Z
Bryan Krause
<p>I'm currently writing a paper and one of the points is about how overprotective parenting can cause anxiety in children.</p>
<p>The results seem pretty clear cut in favour of the assertion (1).
<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5007197/" rel="nofollow noreferrer">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5007197/</a></p>
<p>However, there are individual studies (far fewer than those suggesting a link) that argue the relationship between overprotective parenting and child anxiety isn't really there (2).</p>
<blockquote>
<p>The relationship between overprotective parenting and child anxiety
has been examined repeatedly because theories emphasize its role in
the maintenance of child anxiety. No study has yet tested whether this
relationship is unique to child anxiety, by controlling for commonly
co-occurring behavior problems within the same children. The current
study examined 190 children (age 7-13, 118 [corrected] boys) referred
to mental health clinics and their parents. Results revealed that
significant correlations between overprotective parenting and child
anxiety symptoms disappear after controlling for co-occurring child
behavior symptoms. It appears that overprotection is not uniquely
related to child anxiety. Furthermore, overprotective parenting was
significantly and uniquely related to child behavior symptoms.
Researchers and practitioners need to consider co-occurring child
behavior problems when working with the parents of anxious children.
<a href="https://pubmed.ncbi.nlm.nih.gov/22659077/" rel="nofollow noreferrer">https://pubmed.ncbi.nlm.nih.gov/22659077/</a></p>
</blockquote>
<p>Given all these studies are valid, how would a scientist deal with this? Would the assumption be that there's mixed results and therefore its inconclusive? Or would the assumption be that because the majority is pro, we can view the link as existing?</p>

CC BY-SA 4.0
2022-04-29T20:26:51.443Z
It's worth learning about [p-values](https://en.wikipedia.org/wiki/P-value).  A certain number of negative results is *expected*, and in fact, if it isn't found in the literature, then that is a clear sign of [publication bias](https://en.wikipedia.org/wiki/Publication_bias).
Arnon Weinberg
<statistics>
How do scientists/psychologists deal with conflicting experiment results?
Jim stoke
# How do scientists/psychologists deal with conflicting experiment results?

(Asked by: username_0 on 2022-04-29)

I'm currently writing a paper and one of the points is about how overprotective parenting can cause anxiety in children.

The results seem pretty clear cut in favour of the assertion (1).
<https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5007197/>

However, there are individual studies (far fewer than those suggesting a link) that argue the relationship between overprotective parenting and child anxiety isn't really there (2).

> The relationship between overprotective parenting and child anxiety
> has been examined repeatedly because theories emphasize its role in
> the maintenance of child anxiety. No study has yet tested whether this
> relationship is unique to child anxiety, by controlling for commonly
> co-occurring behavior problems within the same children. The current
> study examined 190 children (age 7-13, 118 [corrected] boys) referred
> to mental health clinics and their parents. Results revealed that
> significant correlations between overprotective parenting and child
> anxiety symptoms disappear after controlling for co-occurring child
> behavior symptoms. It appears that overprotection is not uniquely
> related to child anxiety. Furthermore, overprotective parenting was
> significantly and uniquely related to child behavior symptoms.
> Researchers and practitioners need to consider co-occurring child
> behavior problems when working with the parents of anxious children.
> <https://pubmed.ncbi.nlm.nih.gov/22659077/>

Given all these studies are valid, how would a scientist deal with this? Would the assumption be that there's mixed results and therefore its inconclusive? Or would the assumption be that because the majority is pro, we can view the link as existing?

**Tags:** statistics

> **Comments:**
> **username_1**: It's worth learning about [p-values](https://en.wikipedia.org/wiki/P-value).  A certain number of negative results is *expected*, and in fact, if it isn't found in the literature, then that is a clear sign of [publication bias](https://en.wikipedia.org/wiki/Publication_bias).  

---

## 1 answers

### Answer by username_2 on 2022-04-29. Score: 4 | Accepted answer

[Meta-analysis](https://en.wikipedia.org/wiki/Meta-analysis) is the statistical practice of combining results across studies in a principled way. Meta-analyses weight the relative strength of evidence from different studies to generate an overall conclusion.

However, there is a bit of art to it, as well, as it's rare that studies are truly fully compatible with one another. They may be in different study populations (for example, sampling from different countries or using different inclusion/exclusion criteria), use different outcome measures, etc. Generally, individual studies also talk about their limitations, for example in the second paper you reference, the authors write about their sample:

> The sample was recruited for a treatment trial for anxiety disorders... Compared to a ... representative sample ... the average score on Internalizing problems was significantly higher for the
> children in the current study. ...recruitment strategies might also have caused the
> number of children with separation anxiety disorder, social phobia and generalized anxiety disorder to be especially high in our
> sample...

And about their outcome measure:

> In past research, overprotective parenting has been assessed
> using different methods and overprotection is only one of several
> parenting dimensions that have been considered. The results of the
> current study need to be replicated using different measurements
> of overprotective parenting and include other parenting dimensions.

Papers presenting meta analyses often include a qualitative review of the literature, as well, and others may also write [review articles](https://en.wikipedia.org/wiki/Review_article) that summarize a research field without doing a formal meta analysis. These papers will consider various explanations for differences in study results, which can include random statistical variation as well as systematic methodological differences. For example, in the second article you quote from, emphasis mine:

> Results revealed that significant correlations between overprotective parenting and child anxiety symptoms **disappear after controlling for co-occurring child behavior symptoms**

The authors here are describing a departure in their research from what others have done: they've adjusted for "child behavior symptoms". However, I think they are not really concluding what you think: it isn't that they are saying that there is no link between overprotective parenting and child anxiety, they are saying that overprotective parenting is associated with a wider range of behavioral problems than just anxiety; anxiety isn't unique or special. Because these are all correlational studies (and it isn't really ethical or practically possible to run a randomized experiment where children are assigned to "overprotective" or "not-overprotective" parenting), it's [not really possible to establish causality](https://xkcd.com/552/), so the causality could be in the opposite direction: that children with behavior problems lead their parents to be overprotective.

If you're looking at a research question that has been studied in multiple separate research papers, I would recommend looking for a recent meta analysis or review paper for further guidance in interpreting the literature.

> **Comments:**
> **username_0**: Thanks a lot! I found a meta analysis from 2007, so probably a bit too far back but these types of analysis are definitely something I'll use from now on. What threw me was "child anxiety symptoms disappear after controlling for co-occurring child behavior symptoms" because I interpreted it as saying that overprotection wasn't responsible for anxiety but rather co-occuring behaviours were, and when they were factored out, then there was no relationship between overprotection and anxiety.  
> **username_0**: Just to follow up, I found a good meta-analysis from this year about the relationship of overprotective parents to internalising/externalising, which is a huge predictor of anxiety. Thanks a lot again! https://onlinelibrary.wiley.com/doi/full/10.1111/sode.12590  

---

