<p>Let's start with signal detection methods. The big one in memory literature is <a href="http://www.linguistics.ucla.edu/faciliti/facilities/statistics/dprime.htm" rel="nofollow">D-Prime</a> analysis (hits versus false alarms). If the subject chooses X more times than Y, you can compare the number of times choice X is made correctly versus chosen incorrectly (as a false alarm). If that was your paradigm, then choice Y would be the correct rejection (if correct) or a miss (if incorrect).</p>

<p>With this in mind, you can add some degree of confidence to your paradigm. Instead of only two choices, x or y, you can give for options (1 - definitely x, 2 - probably x, 3 - probably y, 4 - definitely y). Never give the option to say, I don't know, because subjects can be unaware that he or she knows the answer, and studies show subjects can be greater than chance through <em>unconscious recognition</em>. Then, if you're afraid of a side bias, extracting only "definitely" answers should be a way of excluding that possibility, because if they were just guessing they would choose the "probably" answers.</p>

CC BY-SA 3.0
2012-08-11T11:24:10.677Z
True, moving to confidence ratings rather than yes / no would allow exclusion of guesses.  I'd be worried about excluding too many with only the suggested four options, but I agree something along those lines should work.
Charlie
CC BY-SA 3.0
2012-08-11T11:36:43.903Z
I ended up running the experiment with the inclusion of unseen items, and will control for the observed response bias across those (as if no bias they should be selected with equal frequency) with an ANCOVA.
Charlie
CC BY-SA 3.0
2012-08-10T18:24:39.783Z
2012-08-10T18:24:39.783Z
David Winer
<p>I want to present participants a pair of items $a$ and $b$, each with an association $x$ or $y$. Later on participants will presented with the items singularly, and asked  'what was the association?'.  I'm interested in differences in their memory for the association, between levels of the association. </p>

<p>As I've specified it there is no way to separate differences in memory performance between levels from response bias, because being incorrect on association $x$ means that another response for association $y$ is recorded.  Signal detection methods seem to give me an overall measure of discriminability between the two and bias, but can't tell me if the bias is due to genuinely improved memory for $x$, or simply a predisposition to respond $x$.</p>

<p><strong>What is the best way to deal with this?</strong></p>

<p><strong>Initial thoughts:</strong> I'm considering including previously unseen elements into the memory task and getting a measure of response bias that way.  </p>

<p><hr />
Here is some random example data to help with thinking:</p>

<pre><code>choice X | X    choice Y | X   choice Y | Y   choice X | Y
   18               18         18              18    
   24               12         18              18
   30                6         14              22    
   29                7         14              22
</code></pre>

<memory><methodology><experimental-psychology><statistics>
Controlling for response bias in a forced choice memory task
Charlie
# Controlling for response bias in a forced choice memory task

(Asked by: username_0 on 2012-08-03)

I want to present participants a pair of items $a$ and $b$, each with an association $x$ or $y$. Later on participants will presented with the items singularly, and asked  'what was the association?'.  I'm interested in differences in their memory for the association, between levels of the association. 
As I've specified it there is no way to separate differences in memory performance between levels from response bias, because being incorrect on association $x$ means that another response for association $y$ is recorded.  Signal detection methods seem to give me an overall measure of discriminability between the two and bias, but can't tell me if the bias is due to genuinely improved memory for $x$, or simply a predisposition to respond $x$.
What is the best way to deal with this?
Initial thoughts: I'm considering including previously unseen elements into the memory task and getting a measure of response bias that way.  

Here is some random example data to help with thinking:
choice X | X    choice Y | X   choice Y | Y   choice X | Y
   18               18         18              18    
   24               12         18              18
   30                6         14              22    
   29                7         14              22

**Tags:** memory, methodology, experimental-psychology, statistics


---

## 1 answers

### Answer by username_1 on 2012-08-10. Score: 4

Let's start with signal detection methods. The big one in memory literature is [D-Prime](http://www.linguistics.ucla.edu/faciliti/facilities/statistics/dprime.htm) analysis (hits versus false alarms). If the subject chooses X more times than Y, you can compare the number of times choice X is made correctly versus chosen incorrectly (as a false alarm). If that was your paradigm, then choice Y would be the correct rejection (if correct) or a miss (if incorrect).

With this in mind, you can add some degree of confidence to your paradigm. Instead of only two choices, x or y, you can give for options (1 - definitely x, 2 - probably x, 3 - probably y, 4 - definitely y). Never give the option to say, I don't know, because subjects can be unaware that he or she knows the answer, and studies show subjects can be greater than chance through *unconscious recognition*. Then, if you're afraid of a side bias, extracting only "definitely" answers should be a way of excluding that possibility, because if they were just guessing they would choose the "probably" answers.

> **Comments:**
> **username_0**: True, moving to confidence ratings rather than yes / no would allow exclusion of guesses.  I'd be worried about excluding too many with only the suggested four options, but I agree something along those lines should work.  
> **username_0**: I ended up running the experiment with the inclusion of unseen items, and will control for the observed response bias across those (as if no bias they should be selected with equal frequency) with an ANCOVA.  

---

