<p>I don't know if you can completely remove the selection bias from such a sample. </p>

<p>You seem to be referring (in some sense, at least) to regression towards the mean. This is one of the biggest (if not the biggest) problem with much research on clinical samples. </p>

<p>What I would do (assuming resources) would be to take the same size (or larger) sample from the general population, and administer all of the same measures and tests to them. This would allow you to separate the predictors of the clinical outcome versus the variation due to other factors. I would probably employ matching from the clinical sample to the general sample (Gelman and Hill have a great chapter in their book about the matching, and <a href="http://www.amazon.co.uk/Analysis-Regression-Multilevel-Hierarchical-Analytical/dp/052168689X/ref=sr_1_1?ie=UTF8&amp;qid=1340219301&amp;sr=8-1" rel="nofollow">that entire book</a> is filled with helpful advice to analogous problems.</p>

<p>In my research, (on predictors of placebo response) I took large samples from the University where I was conducting my research on all of my self report measures, and compared my experimental population to them to determine if my sample were representative of the larger populatiom, which is a somewhat similiar approach to a common problem. </p>

<p>Unfortunately, other than the Gelman and Hill book, I'm not aware of any literature on the topic (but that may be my own fault). </p>

<p>I hope this helps. </p>

CC BY-SA 3.0
2012-06-20T19:11:50.703Z
2012-06-20T19:11:50.703Z
richiemorrisroe
<h3>Context</h3>

<p>I sometimes speak to researchers studying the relationship between variables in clinical psychology samples.
A typical example is as follows:</p>

<ul>
<li>100 patients with diagnosed depression</li>
<li>many clinically relevant variables are measured such as demographics, anxiety, intelligence, drug and alcohol use, and so on.</li>
</ul>

<p>Researchers often want to then develop models of what predicts depression.</p>

<p>However, the problem is that the sample was selected because they already have been deemed to possess a certain threshold level of depression.
Substantial variation still remains in depression with some participants being more severe than others.</p>

<p>Thus, there are problems in trying to generalise observed relationships to describe what predicts depression, because it is not a random sample of the population.</p>

<p>I use depression as a specific case, but the problem applies to many studies of clinical populations (e.g., kids with behavioural problems, kids with intellectual disabilities, OCD, etc.).</p>

<h3>Question</h3>

<ul>
<li>What advice would you give to such researchers about how to analyse and generalise from such data?</li>
<li>Are there any references that provide an example of best practice of how to analyse and generalise from such data?</li>
</ul>

<methodology><abnormal-psychology><experimental-psychology><statistics>
Inferring relationships between psychological variables in clinical samples and issues in generalisation
Jeromy Anglim
# Inferring relationships between psychological variables in clinical samples and issues in generalisation

(Asked by: username_0 on 2011-06-06)

### Context

I sometimes speak to researchers studying the relationship between variables in clinical psychology samples.
A typical example is as follows:

* 100 patients with diagnosed depression
* many clinically relevant variables are measured such as demographics, anxiety, intelligence, drug and alcohol use, and so on.

Researchers often want to then develop models of what predicts depression.

However, the problem is that the sample was selected because they already have been deemed to possess a certain threshold level of depression.
Substantial variation still remains in depression with some participants being more severe than others.

Thus, there are problems in trying to generalise observed relationships to describe what predicts depression, because it is not a random sample of the population.

I use depression as a specific case, but the problem applies to many studies of clinical populations (e.g., kids with behavioural problems, kids with intellectual disabilities, OCD, etc.).

### Question

* What advice would you give to such researchers about how to analyse and generalise from such data?
* Are there any references that provide an example of best practice of how to analyse and generalise from such data?

**Tags:** methodology, abnormal-psychology, experimental-psychology, statistics


---

## 1 answers

### Answer by username_1 on 2012-06-20. Score: 4

I don't know if you can completely remove the selection bias from such a sample.

You seem to be referring (in some sense, at least) to regression towards the mean. This is one of the biggest (if not the biggest) problem with much research on clinical samples.

What I would do (assuming resources) would be to take the same size (or larger) sample from the general population, and administer all of the same measures and tests to them. This would allow you to separate the predictors of the clinical outcome versus the variation due to other factors. I would probably employ matching from the clinical sample to the general sample (Gelman and Hill have a great chapter in their book about the matching, and [that entire book](http://www.amazon.co.uk/Analysis-Regression-Multilevel-Hierarchical-Analytical/dp/052168689X/ref=sr_1_1?ie=UTF8&qid=1340219301&sr=8-1) is filled with helpful advice to analogous problems.

In my research, (on predictors of placebo response) I took large samples from the University where I was conducting my research on all of my self report measures, and compared my experimental population to them to determine if my sample were representative of the larger populatiom, which is a somewhat similiar approach to a common problem.

Unfortunately, other than the Gelman and Hill book, I'm not aware of any literature on the topic (but that may be my own fault).

I hope this helps.


---

