<p><strong>Using Parameters Estimated from an Individual in a Group Analysis</strong></p>

<p>In a way this is exactly what usually happens when we calculate the mean reaction time across all conditions for a group of participants. When we normally calculate mean reaction times we assume that some process (P) takes t milliseconds to complete plus some Gaussian distributed noise. We want to estimate this parameter (t) across conditions, to make some statement about how the treatment effects processing. Then we calculate this set of parameters for a number of participants to ensure that we are looking at something interesting at the population level.</p>

<p>However, we can adopt more sophisticated models with more parameters that better represent how reaction times actually vary. Take the simple case of the ex-Gaussian distribution. This distribution is the combination of an exponential distribution and a Gaussian distribution. The shape of an ex-Gaussian is much closer to a distribution of observed reaction times than just the Gaussian alone. Reaction times modelled with an ex-Gaussian involve fitting two parameters; one for the Gaussian and one for the exponential components of the model. So in this case we would calculate these parameters for each condition and participant. Then we would enter these sets of parameters into two repeated measures ANOVAs; one for the Gaussian parameters and one for the exponential parameters. Then we can assess whether the treatment alters how the reaction times vary. The problem with the ex-Gaussian is that the two parameters have no psychological basis, so there is no meaningful interpretation of the treatment effect on these parameters.</p>

<p>This is where diffusion models are useful because the parameters in the model are psychologically meaningful, allowing us to make interesting inferences from changes in the parameters across conditions.</p>

<p><strong>Same Model or the Best Model or each Participant?</strong></p>

<p>I'm no expert with diffusion models, but I would expect that the same model should be applied to each participant. Not doing so would mean that you are assuming a different model for your reaction time distributions across participants.</p>

CC BY-SA 3.0
2012-10-30T21:21:33.000Z
2012-10-30T21:21:33.000Z
Bronson
<p><a href="http://ski.clps.brown.edu/hddm_docs/" rel="nofollow">HDDM</a> allows to fit a <em>hierarchical</em> drift diffusion model and gives you the posterior of the group effects.</p>

<p>That means, the model is fit for all subjects simultaneously. Subjects are allowed to vary, but an underlying shared effect is assumed (and the same model is fit for all subjects, although of course a parameter may end up as 0 for a subject). The model is Bayesian and the results are directly interpretable as Bayesian inference.</p>

<p>It's in Python.</p>

CC BY-SA 3.0
2015-10-24T16:37:51.670Z
2015-10-24T16:37:51.670Z
jona
<p>If you are planning on comparing the parameters, then you should apply the same model. However with only 80 observations per session per subjects you might need to use hierarchical bayesian framework, because sequential sampling models are pretty unstbale with so few observations.</p>

<p>Here is a nice paper on combing RT models with neural data using hierarchical bayesian framework:
<a href="http://www.newcl.org/publications/TurForWagBroSedSte13.pdf" rel="nofollow">http://www.newcl.org/publications/TurForWagBroSedSte13.pdf</a></p>

CC BY-SA 3.0
2015-10-20T16:24:46.427Z
2015-10-20T16:24:46.427Z
Mabu
<p>I would like to analyze a dataset in which subjects' reaction time and error rate have been recorded. In order to account for potential speed-accuracy trade-off I am planning to make use of the <a href="https://cogsci.stackexchange.com/questions/1773/is-there-an-r-implementation-of-the-linear-ballistic-accumulator-model-or-ratcli/1774#1774">Ratcliff's diffusion model</a>. My data is a within-subject repeated-measures design with two sessions per subject (drug &amp; placebo) with multiple measurements in each session (around 80).</p>

<p>While reading some papers regarding this topic I found a nice report by <a href="http://www.jneurosci.org/content/31/4/1254.full.pdf+html?sid=f1b59b47-53d5-409e-b43e-c56836f4987f" rel="nofollow noreferrer">Wenzlaff et al. (2011)</a>. First they fit the diffusion model for every subject and then take the estimated parameters to an ANOVA. However I was confused by their approach: it seems they select the best fitting model for every subject and in this way are fitting models with different parameters for the subjects. My understanding was that it is not a valid approach to put together estimates from models with different parameters?</p>

<p>My questions are:</p>

<ul>
<li>Would it be a valid approach to first fit the diffusion model
separately for every subject and then take the estimated parameters
to a group-level analysis (e.g. repeated-measures ANOVA)?</li>
<li>Prior to ANOVA of parameter estimates: is it - in the above context -
better to fit the same model to all subjects or to select the best
fitting model for every subject?</li>
</ul>

<blockquote>
  <p>Wenzlaff, H., Bauer, M., Maess, B., &amp; Heekeren, H. R. (2011). Neural characterization of the speed–accuracy tradeoff in a perceptual decision-making task. The Journal of Neuroscience, 31(4), 1254-1266.</p>
</blockquote>

CC BY-SA 3.0
2012-10-30T12:19:37.117Z
Nice question. Your account is currently unregistered, and if your cookies are cleared then you will lose access to it. You have made great contributions to this site, and I highly recommend registering your account. :D
Artem Kaznatcheev
CC BY-SA 3.0
2012-10-30T21:01:59.940Z
"My understanding was that it is not a valid approach to put together estimates from models with different parameters" - Where did you understand that from?
Ofri Raviv
CC BY-SA 3.0
2012-10-30T21:03:39.927Z
Is there anything specific to Ratcliff's model here? Because the way I see it, this question is equally valid to any model you fit to any multi-subject data.
Ofri Raviv
CC BY-SA 3.0
2012-11-13T23:36:09.713Z
Do I understand correctly that you are planning to fit the diffusion model on a by-participant basis to two data points per participant?
Pavel
CC BY-SA 3.0
2012-11-14T07:23:16.657Z
No - sorry for the unclear question. It is two sessions (drug & placebo) with several measurements in each session. I updated the question.
jokel
<methodology><statistics><reaction-time>
Analysis of parameter estimates from Ratcliff diffusion model
jokel
# Analysis of parameter estimates from Ratcliff diffusion model

(Asked by: username_0 on 2012-10-30)

I would like to analyze a dataset in which subjects' reaction time and error rate have been recorded. In order to account for potential speed-accuracy trade-off I am planning to make use of the [Ratcliff's diffusion model](https://cogsci.stackexchange.com/questions/1773/is-there-an-r-implementation-of-the-linear-ballistic-accumulator-model-or-ratcli/1774#1774). My data is a within-subject repeated-measures design with two sessions per subject (drug & placebo) with multiple measurements in each session (around 80).

While reading some papers regarding this topic I found a nice report by [Wenzlaff et al. (2011)](http://www.jneurosci.org/content/31/4/1254.full.pdf+html?sid=f1b59b47-53d5-409e-b43e-c56836f4987f). First they fit the diffusion model for every subject and then take the estimated parameters to an ANOVA. However I was confused by their approach: it seems they select the best fitting model for every subject and in this way are fitting models with different parameters for the subjects. My understanding was that it is not a valid approach to put together estimates from models with different parameters?

My questions are:

* Would it be a valid approach to first fit the diffusion model
  separately for every subject and then take the estimated parameters
  to a group-level analysis (e.g. repeated-measures ANOVA)?
* Prior to ANOVA of parameter estimates: is it - in the above context -
  better to fit the same model to all subjects or to select the best
  fitting model for every subject?

> Wenzlaff, H., Bauer, M., Maess, B., & Heekeren, H. R. (2011). Neural characterization of the speed–accuracy tradeoff in a perceptual decision-making task. The Journal of Neuroscience, 31(4), 1254-1266.

**Tags:** methodology, statistics, reaction-time

> **Comments:**
> **username_1**: Nice question. Your account is currently unregistered, and if your cookies are cleared then you will lose access to it. You have made great contributions to this site, and I highly recommend registering your account. :D  
> **username_2**: "My understanding was that it is not a valid approach to put together estimates from models with different parameters" - Where did you understand that from?  
> **username_2**: Is there anything specific to Ratcliff's model here? Because the way I see it, this question is equally valid to any model you fit to any multi-subject data.  
> **username_3**: Do I understand correctly that you are planning to fit the diffusion model on a by-participant basis to two data points per participant?  
> **username_0**: No - sorry for the unclear question. It is two sessions (drug & placebo) with several measurements in each session. I updated the question.  

---

## 3 answers

### Answer by username_4 on 2012-10-30. Score: 6

**Using Parameters Estimated from an Individual in a Group Analysis**

In a way this is exactly what usually happens when we calculate the mean reaction time across all conditions for a group of participants. When we normally calculate mean reaction times we assume that some process (P) takes t milliseconds to complete plus some Gaussian distributed noise. We want to estimate this parameter (t) across conditions, to make some statement about how the treatment effects processing. Then we calculate this set of parameters for a number of participants to ensure that we are looking at something interesting at the population level.

However, we can adopt more sophisticated models with more parameters that better represent how reaction times actually vary. Take the simple case of the ex-Gaussian distribution. This distribution is the combination of an exponential distribution and a Gaussian distribution. The shape of an ex-Gaussian is much closer to a distribution of observed reaction times than just the Gaussian alone. Reaction times modelled with an ex-Gaussian involve fitting two parameters; one for the Gaussian and one for the exponential components of the model. So in this case we would calculate these parameters for each condition and participant. Then we would enter these sets of parameters into two repeated measures ANOVAs; one for the Gaussian parameters and one for the exponential parameters. Then we can assess whether the treatment alters how the reaction times vary. The problem with the ex-Gaussian is that the two parameters have no psychological basis, so there is no meaningful interpretation of the treatment effect on these parameters.

This is where diffusion models are useful because the parameters in the model are psychologically meaningful, allowing us to make interesting inferences from changes in the parameters across conditions.

**Same Model or the Best Model or each Participant?**

I'm no expert with diffusion models, but I would expect that the same model should be applied to each participant. Not doing so would mean that you are assuming a different model for your reaction time distributions across participants.


---

### Answer by username_5 on 2015-10-24. Score: 1

[HDDM](http://ski.clps.brown.edu/hddm_docs/) allows to fit a *hierarchical* drift diffusion model and gives you the posterior of the group effects.

That means, the model is fit for all subjects simultaneously. Subjects are allowed to vary, but an underlying shared effect is assumed (and the same model is fit for all subjects, although of course a parameter may end up as 0 for a subject). The model is Bayesian and the results are directly interpretable as Bayesian inference.

It's in Python.


---

### Answer by username_6 on 2015-10-20. Score: 0

If you are planning on comparing the parameters, then you should apply the same model. However with only 80 observations per session per subjects you might need to use hierarchical bayesian framework, because sequential sampling models are pretty unstbale with so few observations.

Here is a nice paper on combing RT models with neural data using hierarchical bayesian framework:
<http://www.newcl.org/publications/TurForWagBroSedSte13.pdf>


---

