<p>What you are looking for is called Hierarchical, Multi-level or Random-effects model. In your particular case the solution is a hierarchical logistic regression. </p>

<p>Assume $y_{st} \in \{0,1\}$ is the response of subject $s$ on trial $t$ and $x$ is the dependent variable then a simple hierarchical model that solves your problem is: </p>

<p>$y_{st}\sim \mathrm{Bernoulli}(\mathrm{logit}(\alpha_s+\beta_s x))$</p>

<p>$\beta_s \sim \mathcal{N}(\mu,\sigma)$</p>

<p>where $\mu$ is the population value of the slope and $\beta_s$ is the subject level estimate. Roughly, $\mu$ is a weighted average of all $\beta_s$ where the weight of each $\beta_s$ is inversely proportional to the variance of the estimate of $\beta_s$. For more details on hierarchical logistic regression and for extensions of the simple model, that I have suggested above, refer to Chapter 14 in Gelman &amp; Hill (2006).</p>

<blockquote>
  <p>Fitting a function to this data, the slope should be nearly flat, right? </p>
</blockquote>

<p>No. The slope should be <em>uncertain</em>. Flat slope looks differently, say $(10,0.61), (20,0.59),(30,0.6),(40,0.58),(50,0.6)$. The corresponding estimate of $\beta$ should show wide interval such that you can't conclude that $\beta&gt;0$ or $\beta&lt;0$ or  $\beta=0$ (as you suggested).</p>

<p>How will a hierarchical model handle such uncertain $\beta_s$? This $\beta_s$ will contribute little to the estimate of $\mu$. Instead $\beta_s$ for this particular subject will be pulled towards $\mu$. The hierarchical model will effectively tell you that if your data is inconclusive, it will just assume that the subject is a typical member of the population (that is if $\mu$ had been estimated reliably) and discard the erratic data.</p>

<p>Literature: 
Gelman, A., &amp; Hill, J. (2006). <em>Data analysis using regression and multilevel/hierarchical models</em>. Cambridge University Press.</p>

CC BY-SA 3.0
2015-06-19T18:34:29.953Z
2015-06-19T18:34:29.953Z
matus
<p>At the heart of the matter is the fact that 60% "yes" responses independent of the stimulus level (i.e., the problematic data) can arise from both an extremely sensitive subject (i.e., steep slope) with a moderate bias and a high lapse rate and an extremely insensitive subject (i.e., shallow slope) with a moderate bias and a low lapse rate. For your data the steep slope/high lapse rate fit is slightly better than the shallow slope/low lapse rate when your prior on the lapse rate is based on a Beta distribution. My guess is if you used a uniform prior on the lapse rate, and possibly on the guessing rate, this will result in the shallow slope fit being better. I would try something like "Uniform(0,0.1)".</p>

CC BY-SA 3.0
2014-09-10T13:44:12.493Z
2014-09-10T13:44:12.493Z
StrongBad
<p>I'm fitting a psychometric function to a range of data. The majority of this data lends itself to a sigmoidal fit (i.e. participants can do the task), but some individuals are absolutely unable to do the task. I'm planning to compare the slopes obtained from different conditions, but I've hit a wall with the unable-to-do-the-task data.</p>

<p><img src="https://i.sstatic.net/XwEJo.png" alt="enter image description here"></p>

<p>Fitting a function to this data, the slope should be nearly flat, right? However, the data is really noisy and some weird fitting is occurring - I end up getting erroneously high slopes. I'm using pypsignifit, the parameters I'm using can be seen below. Any idea how to stop this happening?</p>

<pre><code>num_of_block  = 7
num_of_trials = 20

stimulus_intensities=[3, 7, 13, 20, 27, 32, 39] # stimulus levels
percent_correct=[.38, .75, .6, .43, .7, .65, .43]     # percent correct sessions 1-3
num_observations     = [num_of_trials] * num_of_block      # observations per block
data= np.c_[stimulus_intensities, percent_correct, num_observations]
nafc = 1
constraints = ('unconstrained', 'unconstrained', 'unconstrained', 'Beta(2,20)' )
boot = psi.BootstrapInference ( data, core='ab', sigmoid='gauss', priors=constraints, nafc=nafc )
boot.sample(2000)
print 'pse', boot.getThres(0.5)
print 'slope', boot.getSlope()
print 'jnd', (boot.getThres(0.75)-boot.getThres(0.25))
</code></pre>

CC BY-SA 3.0
2018-01-07T07:20:50.890Z
I'm curious if you've solved the problem in the meantime and how did you finally approach it.
got trolled too much this week
CC BY-SA 3.0
2012-06-03T17:09:38.447Z
The data look really like the participant performed on chance level. Actually, I would not try to fit them at all, because the fits cannot get better than your example.
H.Muster
CC BY-SA 3.0
2012-06-03T18:09:55.490Z
That's exactly what happened. It may be counterintuitive to try to fit the data, but I really want to be able to compare the slope of these participants with ones who performed better using something like a t-test. Hence the **'need'** to fit.
luser
CC BY-SA 3.0
2012-06-03T19:30:23.513Z
Then I would try different psychometric functions (e.g, logistic, Weibull) until I find one that is fitted to the data as a straight line with a slope of almost zero.
H.Muster
CC BY-SA 3.0
2012-06-03T20:08:49.973Z
Makes sense, but the main issue I have is I'd have to arbitrarily choose when to use this alternative fit. The same participant appeared to perform at chance on another condition, but the cumulative gaussian fitted well and showed a not-immediately-apparent slope.
luser
CC BY-SA 3.0
2012-06-04T05:56:14.010Z
I would not suggest to use different functions for individual fits, but use one functions for all, i.e., choose the function that gives the best overall solutions. Concerning the bad cases: did you try to seed the estimation routine with starting values nearer to 0.5 for the guessing rate and the lapsing rate?
H.Muster
CC BY-SA 3.0
2012-06-04T13:04:08.243Z
Upon reflection, that was obvious. Sorry! Seeding with values close to 0.5 does not help, nor does changing function - the obscure slope from the first data-point is still there. Ideally I'd just not use that data in my analyses, but seeing as the whole point of this venture is to compare those who are 'good' as the task compared to those who are not...
luser
CC BY-SA 3.0
2012-06-04T14:28:30.877Z
Another idea: try to constraint lapsing rate and guessing rate to values between 0.3 and 0.8 (i.e., the minimum and maximum percent correct rate in the data set your picture is based on).
H.Muster
CC BY-SA 3.0
2012-06-04T15:18:25.497Z
let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/3654/discussion-between-luser-and-h-muster)
luser
CC BY-SA 3.0
2012-06-11T20:03:07.297Z
Is this a Yes/No task, or a 2AFC design?
Ofri Raviv
CC BY-SA 3.0
2013-06-30T15:20:31.140Z
What exactly is the plot showing? Is it data from a single participant or data for an item? Since there are no lables for the axes, what is plotted on the x- and y-axis? It looks like the y-axis could be the probability for getting the right answer since it goes from 0 to 1. Is that correct?
Jens Kouros
<methodology><statistics><data>
Fitting a psychometric function when data does not lend itself to a sigmoidal fit
luser
# Fitting a psychometric function when data does not lend itself to a sigmoidal fit

(Asked by: username_0 on 2012-06-03)

I'm fitting a psychometric function to a range of data. The majority of this data lends itself to a sigmoidal fit (i.e. participants can do the task), but some individuals are absolutely unable to do the task. I'm planning to compare the slopes obtained from different conditions, but I've hit a wall with the unable-to-do-the-task data.

Fitting a function to this data, the slope should be nearly flat, right? However, the data is really noisy and some weird fitting is occurring - I end up getting erroneously high slopes. I'm using pypsignifit, the parameters I'm using can be seen below. Any idea how to stop this happening?
num_of_block  = 7
num_of_trials = 20

stimulus_intensities=[3, 7, 13, 20, 27, 32, 39] # stimulus levels
percent_correct=[.38, .75, .6, .43, .7, .65, .43]     # percent correct sessions 1-3
num_observations     = [num_of_trials] * num_of_block      # observations per block
data= np.c_[stimulus_intensities, percent_correct, num_observations]
nafc = 1
constraints = ('unconstrained', 'unconstrained', 'unconstrained', 'Beta(2,20)' )
boot = psi.BootstrapInference ( data, core='ab', sigmoid='gauss', priors=constraints, nafc=nafc )
boot.sample(2000)
print 'pse', boot.getThres(0.5)
print 'slope', boot.getSlope()
print 'jnd', (boot.getThres(0.75)-boot.getThres(0.25))

**Tags:** methodology, statistics, data

> **Comments:**
> **username_2**: The data look really like the participant performed on chance level. Actually, I would not try to fit them at all, because the fits cannot get better than your example.  
> **username_0**: That's exactly what happened. It may be counterintuitive to try to fit the data, but I really want to be able to compare the slope of these participants with ones who performed better using something like a t-test. Hence the **'need'** to fit.  
> **username_2**: Then I would try different psychometric functions (e.g, logistic, Weibull) until I find one that is fitted to the data as a straight line with a slope of almost zero.  
> **username_0**: Makes sense, but the main issue I have is I'd have to arbitrarily choose when to use this alternative fit. The same participant appeared to perform at chance on another condition, but the cumulative gaussian fitted well and showed a not-immediately-apparent slope.  
> **username_2**: I would not suggest to use different functions for individual fits, but use one functions for all, i.e., choose the function that gives the best overall solutions. Concerning the bad cases: did you try to seed the estimation routine with starting values nearer to 0.5 for the guessing rate and the lapsing rate?  
> **username_0**: Upon reflection, that was obvious. Sorry! Seeding with values close to 0.5 does not help, nor does changing function - the obscure slope from the first data-point is still there. Ideally I'd just not use that data in my analyses, but seeing as the whole point of this venture is to compare those who are 'good' as the task compared to those who are not...  
> **username_2**: Another idea: try to constraint lapsing rate and guessing rate to values between 0.3 and 0.8 (i.e., the minimum and maximum percent correct rate in the data set your picture is based on).  
> **username_0**: let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/3654/discussion-between-username_0-and-h-muster)  
> **username_3**: Is this a Yes/No task, or a 2AFC design?  
> **username_4**: What exactly is the plot showing? Is it data from a single participant or data for an item? Since there are no lables for the axes, what is plotted on the x- and y-axis? It looks like the y-axis could be the probability for getting the right answer since it goes from 0 to 1. Is that correct?  
> **username_1**: I'm curious if you've solved the problem in the meantime and how did you finally approach it.  

---

## 2 answers

### Answer by username_5 on 2015-06-19. Score: 3

What you are looking for is called Hierarchical, Multi-level or Random-effects model. In your particular case the solution is a hierarchical logistic regression.

Assume $y\_{st} \in \{0,1\}$ is the response of subject $s$ on trial $t$ and $x$ is the dependent variable then a simple hierarchical model that solves your problem is:

$y\_{st}\sim \mathrm{Bernoulli}(\mathrm{logit}(\alpha\_s+\beta\_s x))$

$\beta\_s \sim \mathcal{N}(\mu,\sigma)$

where $\mu$ is the population value of the slope and $\beta\_s$ is the subject level estimate. Roughly, $\mu$ is a weighted average of all $\beta\_s$ where the weight of each $\beta\_s$ is inversely proportional to the variance of the estimate of $\beta\_s$. For more details on hierarchical logistic regression and for extensions of the simple model, that I have suggested above, refer to Chapter 14 in Gelman & Hill (2006).

> Fitting a function to this data, the slope should be nearly flat, right?

No. The slope should be *uncertain*. Flat slope looks differently, say $(10,0.61), (20,0.59),(30,0.6),(40,0.58),(50,0.6)$. The corresponding estimate of $\beta$ should show wide interval such that you can't conclude that $\beta>0$ or $\beta<0$ or $\beta=0$ (as you suggested).

How will a hierarchical model handle such uncertain $\beta\_s$? This $\beta\_s$ will contribute little to the estimate of $\mu$. Instead $\beta\_s$ for this particular subject will be pulled towards $\mu$. The hierarchical model will effectively tell you that if your data is inconclusive, it will just assume that the subject is a typical member of the population (that is if $\mu$ had been estimated reliably) and discard the erratic data.

Literature:
Gelman, A., & Hill, J. (2006). *Data analysis using regression and multilevel/hierarchical models*. Cambridge University Press.


---

### Answer by username_6 on 2014-09-10. Score: 1

At the heart of the matter is the fact that 60% "yes" responses independent of the stimulus level (i.e., the problematic data) can arise from both an extremely sensitive subject (i.e., steep slope) with a moderate bias and a high lapse rate and an extremely insensitive subject (i.e., shallow slope) with a moderate bias and a low lapse rate. For your data the steep slope/high lapse rate fit is slightly better than the shallow slope/low lapse rate when your prior on the lapse rate is based on a Beta distribution. My guess is if you used a uniform prior on the lapse rate, and possibly on the guessing rate, this will result in the shallow slope fit being better. I would try something like "Uniform(0,0.1)".


---

