<p><strong>Short answer</strong><br>
Based on the experimental restrictions provided in the question and comments, my best bet would be to include 6 catch trials and to randomly present them from trial 6 and onward. Trials 1 &amp; 2 are reserved for training purposes, trials 3 - 5 I would reserve for above-threshold stimuli.</p>

<p><strong>Background</strong><br>
First a problem breakdown as I understand it from the question and the comments:</p>

<ul>
<li>A Yes/No task is performed using method-of-constant stimuli;</li>
<li>Dynamic stimulus range is known and expected outcomes are available;</li>
<li>The detection rate is important, whereas a 50% threshold or psychometric curve are not of interest;</li>
<li>Zero-stimulus (<a href="http://psychologydictionary.org/catch-trial/" rel="nofollow noreferrer">'catch'</a>) trials are a necessary component to validate the method in the normal-group;</li>
<li>No pilot experiment can be performed to validate the methods in a small group of controls;</li>
<li>Two practice trials are needed (stimulus-tone alone; noise-alone);</li>
<li>Critical bottle neck: 20 trials per run</li>
</ul>

<p>So - here is my suggestion, based on my experiences.</p>

<p>First off, in a yes/no task there's anything possible from correct rates from 0% to 100%. To have a reasonably estimate of the correct rate at a certain constant stimulus, multiple trials are necessary. With one trial you only get 0% or 100% correct - such a binary approach may make sense, but it will add substantial jitter. With four trials, you basically end up with correct-rate increments of 25%. Not too bad, but not pretty either. One subject error and the outcome will be substantially affected. With 8 trials, you are left with steps of 12.5%. From my point of view, <strong>8 trials</strong> is the least you should have to have a reasonable estimate with some precision and to allow the subject to make an error without too much of an affect on the outcome. </p>

<p>From a more practical standpoint, you are left with 18 trials per run per subject (?), namely 20 trials minus two practice runs. At the minimum, from my stand point, you should use 3 stimulus levels: zero-intensity (catch trials) and two above detection-threshold stimuli to spread your chances of finding a statistical difference. I would recommend one far-above detection threshold ('High' condition), where also your impaired group will reach correct rates >50% and one low level ('Low) where your impaired group will score &lt;50% to have a condition that will make a positive outcome (difference control-impaired) guaranteed.</p>

<p>This means 18 trials for three conditions: 'Catch'; 'Low'; 'High'. That means 6 trials per condition, leaving you with correct-rate increments of 16.7%. Not too bad.   </p>

<p>Now, where to put those catch trials into the picture?</p>

<p>I would personally enforce the paradigm to start with the two practice trials (first the tone, then noise) and then have at least the first two or three trials above threshold; best thing to do is have the 'High' condition first for two times, then the 'Low' and start randomizing from there. Again, this is all personal opinion, based on my experience that <strong>dis-ambiguating the test protocol</strong> is paramount. There's nothing worse than a confused subject. By having 2 well-above detection threshold stimuli (the same as training trial #1 I guess) the task is clear cut. If you start off with a catch trial after training they will likely be confused ('I didn't hear a tone!?' ; 'Should I press the button?').   </p>

CC BY-SA 3.0
2017-04-19T07:57:50.350Z
2017-04-19T09:58:20.620Z
2017-04-19T09:58:20.620Z
AliceD
<p>I am designing a tone-in-noise detection task (specifically a high frequency N0Spi task). For various reasons I need to use a yes-no paradigm and the method of constant stimuli. I am severely limited on the number of trials that can be presented (total of 20 trials) and the presentation order must be the same for all subjects. The subjects will be completely naive and be unfamiliar with the stimuli.</p>

<p>I was thinking of giving them two "practice" trials: one with a clearly audible signal (if they knew what the signal sounded like) and one that is noise alone. I would provide feedback on these trials in the case of an incorrect response. Is there any literature on if the signal or no signal trial should come first? In a run of 20 trials, is there an optimal number of catch (no signal) trials? Is there an optimal place in the track to put these catch trials?</p>

CC BY-SA 3.0
2017-04-18T20:11:27.933Z
You could think about counter balancing the practice stimuli and see whether the results on the actual trials differ. If they don't, it didn't matter after all. If they do, you have a reliable explanation :)
Robin Kramer-ten Have
CC BY-SA 3.0
2017-04-18T20:16:26.627Z
@RobinKramer in the sane world of well designed research that is piloted before being rolled out in a multi-site study, that is what I would do. In the world of design a test based on current knowledge with no ability to pilot the test before roll out, I was hoping someone else may have done something similar.
StrongBad
CC BY-SA 3.0
2017-04-18T21:00:11.077Z
So with the practice trials that leaves you with 18 trials? Are you planning to construct psychometric curves? Are you really stuck to MOC? My gut feeling says with such a limited number of trials, and if you're stuck to MOC, go for either a small number of points that surely capture the 50% point and forget about catch trials, or go for an adaptive block-design that rapidly changes the dynamic range to the ROI possibly leaving 4 trials for catch trials if you're lucky (steps of 25% correct rate - still too crude to be of any help I guess; 1 mistake of the subject and you're in trouble) +1
AliceD
CC BY-SA 3.0
2017-04-18T21:03:22.943Z
It would be nice if you could tell us if the dynamic range is fixed or that you have really no clue about the outcome; seconding @RobinKramer 's comment. btw: Literature suggests often to include 25% catch trials. If you appreciate an answer with references and all that based on some of the comments here let me know.
AliceD
CC BY-SA 3.0
2017-04-18T21:11:45.180Z
Based on the title proper; I always avoid catch trials at the very beginning of the run, and actually enforce my script to deliver above-threshold trials to disambiguate the test. I'm actually unsure if your question pertains to  "should I" or "when do I"?
AliceD
CC BY-SA 3.0
2017-04-18T21:27:09.977Z
@AliceD I need to stick with MOC. I don't need to be able to create a psychometric function. Their is a wide range of signal levels for which normals will respond nearly 100% of the time and impaired individuals will essentially never respond. To validate the test is working we need some trials that the normals do not respond (i.e., the catch trials) and that the impaired do respond.
StrongBad
CC BY-SA 3.0
2017-04-18T21:28:25.567Z
@AliceD In the absence of knowing what the signal and no signal stimuli sound like, it seems like they need to be given an example of both at the beginning.
StrongBad
CC BY-SA 3.0
2017-08-15T07:09:45.857Z
Aha this question is a prelude to your latest one? Wow, why are you making your life so hard on yourself?
AliceD
<measurement><methodology><psychophysics><experiment-design>
At what time in a psychophysical test are catch trials preferably presented?
StrongBad
# At what time in a psychophysical test are catch trials preferably presented?

(Asked by: username_0 on 2017-04-18)

I am designing a tone-in-noise detection task (specifically a high frequency N0Spi task). For various reasons I need to use a yes-no paradigm and the method of constant stimuli. I am severely limited on the number of trials that can be presented (total of 20 trials) and the presentation order must be the same for all subjects. The subjects will be completely naive and be unfamiliar with the stimuli.

I was thinking of giving them two "practice" trials: one with a clearly audible signal (if they knew what the signal sounded like) and one that is noise alone. I would provide feedback on these trials in the case of an incorrect response. Is there any literature on if the signal or no signal trial should come first? In a run of 20 trials, is there an optimal number of catch (no signal) trials? Is there an optimal place in the track to put these catch trials?

**Tags:** measurement, methodology, psychophysics, experiment-design

> **Comments:**
> **username_1**: You could think about counter balancing the practice stimuli and see whether the results on the actual trials differ. If they don't, it didn't matter after all. If they do, you have a reliable explanation :)  
> **username_0**: @RobinKramer in the sane world of well designed research that is piloted before being rolled out in a multi-site study, that is what I would do. In the world of design a test based on current knowledge with no ability to pilot the test before roll out, I was hoping someone else may have done something similar.  
> **username_2**: So with the practice trials that leaves you with 18 trials? Are you planning to construct psychometric curves? Are you really stuck to MOC? My gut feeling says with such a limited number of trials, and if you're stuck to MOC, go for either a small number of points that surely capture the 50% point and forget about catch trials, or go for an adaptive block-design that rapidly changes the dynamic range to the ROI possibly leaving 4 trials for catch trials if you're lucky (steps of 25% correct rate - still too crude to be of any help I guess; 1 mistake of the subject and you're in trouble) +1  
> **username_2**: It would be nice if you could tell us if the dynamic range is fixed or that you have really no clue about the outcome; seconding @RobinKramer 's comment. btw: Literature suggests often to include 25% catch trials. If you appreciate an answer with references and all that based on some of the comments here let me know.  
> **username_2**: Based on the title proper; I always avoid catch trials at the very beginning of the run, and actually enforce my script to deliver above-threshold trials to disambiguate the test. I'm actually unsure if your question pertains to  "should I" or "when do I"?  
> **username_0**: @username_2 I need to stick with MOC. I don't need to be able to create a psychometric function. Their is a wide range of signal levels for which normals will respond nearly 100% of the time and impaired individuals will essentially never respond. To validate the test is working we need some trials that the normals do not respond (i.e., the catch trials) and that the impaired do respond.  
> **username_0**: @username_2 In the absence of knowing what the signal and no signal stimuli sound like, it seems like they need to be given an example of both at the beginning.  
> **username_2**: Aha this question is a prelude to your latest one? Wow, why are you making your life so hard on yourself?  

---

## 1 answers

### Answer by username_2 on 2017-04-19. Score: 2

**Short answer**  
Based on the experimental restrictions provided in the question and comments, my best bet would be to include 6 catch trials and to randomly present them from trial 6 and onward. Trials 1 & 2 are reserved for training purposes, trials 3 - 5 I would reserve for above-threshold stimuli.

**Background**  
First a problem breakdown as I understand it from the question and the comments:

* A Yes/No task is performed using method-of-constant stimuli;
* Dynamic stimulus range is known and expected outcomes are available;
* The detection rate is important, whereas a 50% threshold or psychometric curve are not of interest;
* Zero-stimulus (['catch'](http://psychologydictionary.org/catch-trial/)) trials are a necessary component to validate the method in the normal-group;
* No pilot experiment can be performed to validate the methods in a small group of controls;
* Two practice trials are needed (stimulus-tone alone; noise-alone);
* Critical bottle neck: 20 trials per run

So - here is my suggestion, based on my experiences.

First off, in a yes/no task there's anything possible from correct rates from 0% to 100%. To have a reasonably estimate of the correct rate at a certain constant stimulus, multiple trials are necessary. With one trial you only get 0% or 100% correct - such a binary approach may make sense, but it will add substantial jitter. With four trials, you basically end up with correct-rate increments of 25%. Not too bad, but not pretty either. One subject error and the outcome will be substantially affected. With 8 trials, you are left with steps of 12.5%. From my point of view, **8 trials** is the least you should have to have a reasonable estimate with some precision and to allow the subject to make an error without too much of an affect on the outcome.

From a more practical standpoint, you are left with 18 trials per run per subject (?), namely 20 trials minus two practice runs. At the minimum, from my stand point, you should use 3 stimulus levels: zero-intensity (catch trials) and two above detection-threshold stimuli to spread your chances of finding a statistical difference. I would recommend one far-above detection threshold ('High' condition), where also your impaired group will reach correct rates >50% and one low level ('Low) where your impaired group will score <50% to have a condition that will make a positive outcome (difference control-impaired) guaranteed.

This means 18 trials for three conditions: 'Catch'; 'Low'; 'High'. That means 6 trials per condition, leaving you with correct-rate increments of 16.7%. Not too bad.

Now, where to put those catch trials into the picture?

I would personally enforce the paradigm to start with the two practice trials (first the tone, then noise) and then have at least the first two or three trials above threshold; best thing to do is have the 'High' condition first for two times, then the 'Low' and start randomizing from there. Again, this is all personal opinion, based on my experience that **dis-ambiguating the test protocol** is paramount. There's nothing worse than a confused subject. By having 2 well-above detection threshold stimuli (the same as training trial #1 I guess) the task is clear cut. If you start off with a catch trial after training they will likely be confused ('I didn't hear a tone!?' ; 'Should I press the button?').


---

