<p>The cross posting make it hard to answer, but I would rather the points here ...</p>

<p>Apart from not running on my machine (something about <code>convert_to_proportions</code> that I don't care to debug or understand) the code <a href="https://stackoverflow.com/questions/50639613/fitting-equal-variance-signal-detection-model-to-roc-data-in-python">you posted</a> seems conceptually correct. Looking at an abridged version of the code</p>

<pre><code>### Input the signal and noise values
## "sure signal", "likely signal", "guess signal", ... ... "sure noise"
signal = [1230, 496, 358, 272, 215, 165]
noise = [111, 216, 349, 540, 625, 895]

# cumulate them
csignal, cnoise = np.cumsum(signal), np.cumsum(noise)

# convert them to proportions and correct so that all but the last elements will never equal 1
propsignal = convert_to_proportions(csignal)
propnoise = convert_to_proportions(cnoise)

# calculate d' for all, except for the last element which is always 1.
d_primes = [get_d_prime(i[0], i[1]) for i in zip(propsignal[0:-1], propnoise[0:-1])]

# calculate c in the same way
cs = [get_c(i[0], i[1]) for i in zip(propsignal[0:-1], propnoise[0:-1])]

#estimate the noise and signal curves. 
#*** This is where my numbers disagree with those in the paper.
estimated_noise = [stats.norm.cdf(-c) for c in cs]
estimated_signal = [stats.norm.cdf(np.mean(d_primes) - c) for c in cs]
</code></pre>

<p>you do not actually need to calculate <code>cs</code>. I suggest instead just do</p>

<pre><code>cs = np.arange(-10, 10, 0.01)
</code></pre>

<p>This way cs varies over the entire range of interest and makes a smooth curve and you avoid issues from interpolating between the points. When you do this, the fitted curve passes very close to the third data point, but in the manuscript the fitted curve passes very close to the fourth data point. The discrepancy between the paper and your plot seems to arise from</p>

<pre><code>stats.norm.cdf(np.mean(d_primes) - c)
</code></pre>

<p>It turns out that <code>np.mean(d_primes)</code> is very close to <code>d_primes[2]</code> (i.e., the $d^\prime$ associated with the third category). If we replace the offending line with</p>

<pre><code>stats.norm.cdf(d_primes[3] - c)
</code></pre>

<p>everything seems to line up.</p>

<p>From a conceptual standpoint, I think that <code>d_primes[3]</code> is the best estimate of $d^\prime$ since that corresponds to the division between possible signal and possible noise on the 6 point rating scale. That said, there may be a method for which <code>np.mean(d_primes)</code> is the best choice. It is also worth noting that the numbers in the excel table in the paper do not match the figure in the paper.</p>

<p>In summary, if you want to get the numbers in the table, use <code>np.mean(d_primes)</code>, but if you want the figure to match use <code>d_primes[3]</code>.</p>

CC BY-SA 4.0
2018-06-04T21:33:41.523Z
Wow, thanks a lot for the help! The reason I chose np.mean(d_primes) is because cell J14 in the author's spreadsheet looks like it is calculated as the mean of the d` array, which they are then using to calculate the estimated measures in the adjacent columns. That said, I happened to notice that by chance - it wasn't explicitly written so it could be wrong. I don't suppose you know of any other resources to guide through this process? I haven't been able to find much.
fffrost
CC BY-SA 4.0
2018-06-04T21:41:56.157Z
@fffrost I changed my mind I like `d_primes[3]` much better than `np.mean(d_primes) ` (see edit). If you want to match the numbers in the table, use the mean, but if you want to match the figure use the 3rd value.
StrongBad
CC BY-SA 4.0
2018-06-04T21:47:45.770Z
Ah that makes a lot of sense (about d_primes[3] being the best estimate of d`), didn't really consider it before. I will go with this rather than the mean, despite the author's value in the table - I guess it's a mistake or something.
fffrost
CC BY-SA 4.0
2018-06-04T20:49:08.260Z
2018-06-04T21:40:57.310Z
2018-06-04T21:40:57.310Z
StrongBad
<p>I posted <a href="https://stackoverflow.com/questions/50639613/fitting-equal-variance-signal-detection-model-to-roc-data-in-python">this question</a> in the programming forums but think I might get some answers if I post here as well... hope that's ok.</p>

<p>I'm trying to follow the steps in <a href="https://www.researchgate.net/publication/47386959_How_to_fit_models_of_recognition_memory_data_using_maximum_likelihood" rel="nofollow noreferrer">this paper</a> to reproduce the excel table shown, but for some reason I'm unable to reproduce the 'estimated' values on the right hand side of the sheet, despite correctly calculating c for each of the six levels. I'm confused as to how the author arrived at those values, so I would appreciate if anyone could point out what's going on.</p>

<memory><psychology><signal-detection>
Fitting equal-variance signal detection model to ROC data
fffrost
# Fitting equal-variance signal detection model to ROC data

(Asked by: username_0 on 2018-06-02)

I posted [this question](https://stackoverflow.com/questions/50639613/fitting-equal-variance-signal-detection-model-to-roc-data-in-python) in the programming forums but think I might get some answers if I post here as well... hope that's ok.

I'm trying to follow the steps in [this paper](https://www.researchgate.net/publication/47386959_How_to_fit_models_of_recognition_memory_data_using_maximum_likelihood) to reproduce the excel table shown, but for some reason I'm unable to reproduce the 'estimated' values on the right hand side of the sheet, despite correctly calculating c for each of the six levels. I'm confused as to how the author arrived at those values, so I would appreciate if anyone could point out what's going on.

**Tags:** memory, psychology, signal-detection


---

## 1 answers

### Answer by username_1 on 2018-06-04. Score: 1 | Accepted answer

The cross posting make it hard to answer, but I would rather the points here ...
Apart from not running on my machine (something about convert_to_proportions that I don't care to debug or understand) the code you posted seems conceptually correct. Looking at an abridged version of the code
### Input the signal and noise values
## "sure signal", "likely signal", "guess signal", ... ... "sure noise"
signal = [1230, 496, 358, 272, 215, 165]
noise = [111, 216, 349, 540, 625, 895]

# cumulate them
csignal, cnoise = np.cumsum(signal), np.cumsum(noise)

# convert them to proportions and correct so that all but the last elements will never equal 1
propsignal = convert_to_proportions(csignal)
propnoise = convert_to_proportions(cnoise)

# calculate d' for all, except for the last element which is always 1.
d_primes = [get_d_prime(i[0], i[1]) for i in zip(propsignal[0:-1], propnoise[0:-1])]

# calculate c in the same way
cs = [get_c(i[0], i[1]) for i in zip(propsignal[0:-1], propnoise[0:-1])]

#estimate the noise and signal curves. 
#*** This is where my numbers disagree with those in the paper.
estimated_noise = [stats.norm.cdf(-c) for c in cs]
estimated_signal = [stats.norm.cdf(np.mean(d_primes) - c) for c in cs]

you do not actually need to calculate cs. I suggest instead just do
cs = np.arange(-10, 10, 0.01)

This way cs varies over the entire range of interest and makes a smooth curve and you avoid issues from interpolating between the points. When you do this, the fitted curve passes very close to the third data point, but in the manuscript the fitted curve passes very close to the fourth data point. The discrepancy between the paper and your plot seems to arise from
stats.norm.cdf(np.mean(d_primes) - c)

It turns out that np.mean(d_primes) is very close to d_primes[2] (i.e., the $d^\prime$ associated with the third category). If we replace the offending line with
stats.norm.cdf(d_primes[3] - c)

everything seems to line up.
From a conceptual standpoint, I think that d_primes[3] is the best estimate of $d^\prime$ since that corresponds to the division between possible signal and possible noise on the 6 point rating scale. That said, there may be a method for which np.mean(d_primes) is the best choice. It is also worth noting that the numbers in the excel table in the paper do not match the figure in the paper.
In summary, if you want to get the numbers in the table, use np.mean(d_primes), but if you want the figure to match use d_primes[3].

> **Comments:**
> **username_0**: Wow, thanks a lot for the help! The reason I chose np.mean(d_primes) is because cell J14 in the author's spreadsheet looks like it is calculated as the mean of the d` array, which they are then using to calculate the estimated measures in the adjacent columns. That said, I happened to notice that by chance - it wasn't explicitly written so it could be wrong. I don't suppose you know of any other resources to guide through this process? I haven't been able to find much.  
> **username_1**: @username_0 I changed my mind I like `d_primes[3]` much better than `np.mean(d_primes) ` (see edit). If you want to match the numbers in the table, use the mean, but if you want to match the figure use the 3rd value.  
> **username_0**: Ah that makes a lot of sense (about d_primes[3] being the best estimate of d`), didn't really consider it before. I will go with this rather than the mean, despite the author's value in the table - I guess it's a mistake or something.  

---

