<p>Regardless of what the chance level exactly was, as alluded to in the comments, the scores leaps from 0.1 to 0.8. This results in nearly vertical slopes that are prone to convergence errors and hence lead to suboptimal threshold estimates in terms of accuracy. MATLAB always warns me when that happens. I have no experience with <em>PsychoPy</em> unfortunately.</p>
<p>If you wish to achieve better accuracies of your threshold estimates of grating contrast levels between 0.035 and 0.04, you simply need more measurements in that range. Above 0.04 there are about 10 redundant points all basically centered at 100% correct. These measurements should be redistributed to the relevant region around threshold instead.</p>
<p>A convenient way of designing such a threshold experiment is using a dynamic procedure that reduces step size along the way, like a staircase procedure <a href="https://asa.scitation.org/doi/pdf/10.1121/1.1912375" rel="nofollow noreferrer">(Levitt, 1971)</a>. Staircase procedures may be less suitable if you wish to generate psychometric curves. If this is the case, using a stair case to swiftly determine thresholds and then choosing a number of fixed gratings below and above the approximate threshold to generate a psychometric curve can help a lot.</p>
<p><sub><strong>Reference</strong><br />
<strong>-</strong> <a href="https://asa.scitation.org/doi/pdf/10.1121/1.1912375" rel="nofollow noreferrer">Levitt, <em>JASA</em> (1971); <strong>49</strong> 467-77</a></sub></p>

CC BY-SA 4.0
2021-08-24T10:01:45.933Z
Thank you, this is very helpful. An adaptive staircase (QUEST) was in fact the first thing that we tried in this project. Pilot participants received their threshold estimates from a one up one down staircase, and then during the yes/no task where signal and noise were shown an equal number of times, everyone's accuracy rate was close to 100%. Are there any common mistakes in implementing the staircase procedure that we might be committing? The staircase was implemented as a yes/no task, can it be that a two-interval forced choice is significantly more reliable in this case?
nat
CC BY-SA 4.0
2021-08-24T10:36:15.990Z
@nat *Are there any common mistakes in implementing the staircase* -- what is the problem exactly?
AliceD
CC BY-SA 4.0
2021-08-24T10:40:17.010Z
Participants achieve near 100% accuracy using the staircase threshold estimate that should result in accuracy rates of 75%, so the threshold estimate from the staircase seems too high
nat
CC BY-SA 4.0
2021-08-24T19:13:35.093Z
@nat on the basis of what do you expect 75%, *i.e.*, what is your golden standard?
AliceD
CC BY-SA 4.0
2021-08-24T21:47:06.080Z
75% performance would be a participant indicating that there was no grating present on about 25% of all trials where the grating was present, and indicating that they saw a grating on 25% of trials where no grating was shown.
nat
CC BY-SA 4.0
2021-08-25T05:49:05.813Z
@nat I know the concepts behind psychophysics,  what I am asking what makes you think it should have been 75%? I don't understand the question-in-comment
AliceD
CC BY-SA 4.0
2021-08-25T08:30:08.933Z
The settings of the adaptive staircase are set to an equivalent of a one up one down staircase
nat
CC BY-SA 4.0
2021-08-25T11:14:18.553Z
@nat then it converges onto 50% not 75% ?
AliceD
CC BY-SA 4.0
2021-08-25T14:58:56.797Z
If minimum accuracy is 50%, it converges to 75%
nat
CC BY-SA 4.0
2023-03-20T10:29:52.870Z
@nat, yes absolutely
AliceD
CC BY-SA 4.0
2021-08-11T07:30:54.503Z
2023-03-20T10:28:37.667Z
2023-03-20T10:28:37.667Z
AliceD
<p>I am in a project where we are preparing a visual decision making experiment which requires participants to detect vertical grating in a patch of dynamic noise (the noise pattern is updated on every frame; the experiment design is partly a replication of a published paper). We need to find signal contrast level that would result in 75% accuracy rate, and after multiple attempts with QUEST and the method of constants, finding this level seems impossible — the thresholds estimated by those two methods result in either close to 100% accuracy, or chance performance. The experiment is implemented in PsychoPy.</p>
<p>What could be the reason for these difficulties finding the &quot;right&quot; contrast level for the grating? The image below is an attempt to fit a psychometric function to the experimental data -- the slope is almost vertical. The horizontal orange line is the 0.75 level, the vertical axis shows the proportion of &quot;yes&quot; responses, and the values on the horizontal axis are contrast levels.</p>
<p><a href="https://i.sstatic.net/qx1Ju.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/qx1Ju.jpg" alt="Curve fit" /></a></p>

CC BY-SA 4.0
2021-08-10T21:11:15.010Z
What is chance accuracy here?
Bryan Krause
CC BY-SA 4.0
2021-08-10T22:40:18.360Z
@BryanKrause 50%
nat
CC BY-SA 4.0
2021-08-10T22:42:44.797Z
Yet on the left of your graph there are several points *far* below chance. I think something is wrong with your experiment.
Bryan Krause
CC BY-SA 4.0
2021-08-10T22:51:32.797Z
@BryanKrause this plot shows data from a titration procedure where signal is present on every trial, so the vertical axis here shows the proportion of "yes" answers. In the "main" experiment the signal (grating) is always shown with the same level of contrast (it is equal to the threshold estimated for each participant individually during the titration), and signal is present on 50% of all trials, so a participant who always presses the "no" button would achieve 50% accuracy.
nat
CC BY-SA 4.0
2021-08-10T22:52:50.933Z
So during the titration, the proportion of "yes" answers can also be as low as 0
nat
CC BY-SA 4.0
2021-08-10T22:52:53.997Z
Then you shouldn't expect chance to be 50% and shouldn't attempt to fit a psychometric curve that assumes chance is 50%.
Bryan Krause
CC BY-SA 4.0
2021-08-10T22:57:39.050Z
@BryanKrause I think you are right! Thank you
nat
<methodology><vision><psychophysics><signal-detection><task-analysis>
Slope of psychometric function too steep?
nat
# Slope of psychometric function too steep?

(Asked by: username_0 on 2021-08-10)

I am in a project where we are preparing a visual decision making experiment which requires participants to detect vertical grating in a patch of dynamic noise (the noise pattern is updated on every frame; the experiment design is partly a replication of a published paper). We need to find signal contrast level that would result in 75% accuracy rate, and after multiple attempts with QUEST and the method of constants, finding this level seems impossible — the thresholds estimated by those two methods result in either close to 100% accuracy, or chance performance. The experiment is implemented in PsychoPy.

What could be the reason for these difficulties finding the "right" contrast level for the grating? The image below is an attempt to fit a psychometric function to the experimental data -- the slope is almost vertical. The horizontal orange line is the 0.75 level, the vertical axis shows the proportion of "yes" responses, and the values on the horizontal axis are contrast levels.

[![Curve fit](https://i.sstatic.net/qx1Ju.jpg)](https://i.sstatic.net/qx1Ju.jpg)

**Tags:** methodology, vision, psychophysics, signal-detection, task-analysis

> **Comments:**
> **username_1**: What is chance accuracy here?  
> **username_0**: @BryanKrause 50%  
> **username_1**: Yet on the left of your graph there are several points *far* below chance. I think something is wrong with your experiment.  
> **username_0**: @BryanKrause this plot shows data from a titration procedure where signal is present on every trial, so the vertical axis here shows the proportion of "yes" answers. In the "main" experiment the signal (grating) is always shown with the same level of contrast (it is equal to the threshold estimated for each participant individually during the titration), and signal is present on 50% of all trials, so a participant who always presses the "no" button would achieve 50% accuracy.  
> **username_0**: So during the titration, the proportion of "yes" answers can also be as low as 0  
> **username_1**: Then you shouldn't expect chance to be 50% and shouldn't attempt to fit a psychometric curve that assumes chance is 50%.  
> **username_0**: @BryanKrause I think you are right! Thank you  

---

## 1 answers

### Answer by username_2 on 2021-08-11. Score: 1

Regardless of what the chance level exactly was, as alluded to in the comments, the scores leaps from 0.1 to 0.8. This results in nearly vertical slopes that are prone to convergence errors and hence lead to suboptimal threshold estimates in terms of accuracy. MATLAB always warns me when that happens. I have no experience with *PsychoPy* unfortuusername_0ely.

If you wish to achieve better accuracies of your threshold estimates of grating contrast levels between 0.035 and 0.04, you simply need more measurements in that range. Above 0.04 there are about 10 redundant points all basically centered at 100% correct. These measurements should be redistributed to the relevant region around threshold instead.

A convenient way of designing such a threshold experiment is using a dynamic procedure that reduces step size along the way, like a staircase procedure [(Levitt, 1971)](https://asa.scitation.org/doi/pdf/10.1121/1.1912375). Staircase procedures may be less suitable if you wish to generate psychometric curves. If this is the case, using a stair case to swiftly determine thresholds and then choosing a number of fixed gratings below and above the approximate threshold to generate a psychometric curve can help a lot.

**Reference**  
**-** [Levitt, *JASA* (1971); **49** 467-77](https://asa.scitation.org/doi/pdf/10.1121/1.1912375)

> **Comments:**
> **username_0**: Thank you, this is very helpful. An adaptive staircase (QUEST) was in fact the first thing that we tried in this project. Pilot participants received their threshold estimates from a one up one down staircase, and then during the yes/no task where signal and noise were shown an equal number of times, everyone's accuracy rate was close to 100%. Are there any common mistakes in implementing the staircase procedure that we might be committing? The staircase was implemented as a yes/no task, can it be that a two-interval forced choice is significantly more reliable in this case?  
> **username_2**: @username_0 *Are there any common mistakes in implementing the staircase* -- what is the problem exactly?  
> **username_0**: Participants achieve near 100% accuracy using the staircase threshold estimate that should result in accuracy rates of 75%, so the threshold estimate from the staircase seems too high  
> **username_2**: @username_0 on the basis of what do you expect 75%, *i.e.*, what is your golden standard?  
> **username_0**: 75% performance would be a participant indicating that there was no grating present on about 25% of all trials where the grating was present, and indicating that they saw a grating on 25% of trials where no grating was shown.  
> **username_2**: @username_0 I know the concepts behind psychophysics,  what I am asking what makes you think it should have been 75%? I don't understand the question-in-comment  
> **username_0**: The settings of the adaptive staircase are set to an equivalent of a one up one down staircase  
> **username_2**: @username_0 then it converges onto 50% not 75% ?  
> **username_0**: If minimum accuracy is 50%, it converges to 75%  
> **username_2**: @username_0, yes absolutely  

---

