<h1>'Precision' in classical test theory</h1>
<p>Most accounts of classical test theory do not have a notion of precision as such, but occasionally, <em>reliability</em> may be called precision instead. The relationship is probably most concisely illustrated with the standard dartboards. This is also explained on the Wikipedia <a href="https://en.wikipedia.org/wiki/Item_response_theory#Information" rel="nofollow noreferrer">Item Response Theory</a> page, but as you can see, in CTT, precision is to reliability what accuracy is to validity.</p>
<p><img src="https://i.sstatic.net/54faT.png" alt="enter image description here" /> <img src="https://i.sstatic.net/hIBe9.png" alt="Tuft U" /></p>
<p><sub>(<a href="https://en.wikipedia.org/wiki/File:Reliability_and_validity.svg" rel="nofollow noreferrer">Wikipedia Reliability article</a> juxtaposed with <a href="https://wikis.uit.tufts.edu/confluence/display/MCM582/2012S01" rel="nofollow noreferrer">a Tufts University guide</a>.</sub></p>
<h2>Origin of 'precision' in classical test theory</h2>
<p>Cronbach (1951) suggested Coombs (1950) as the origin of the reliability/precision confusion.</p>
<blockquote>
<p>Coombs (6) offers the somewhat more satisfactory name &quot;coefficient of precision&quot; for this index which reports the absolute minimum error to be found if the same instrument is applied twice independently to the same subject. A coefficient of stability can be obtained by making the two observations with any desired interval between. A rigorous definition of the coefficient of precision, then, is that it is the limit of the coefficient of stability, as the time between testings becomes infinitesimal.</p>
</blockquote>
<p>I'm not entirely sure if I'm interpreting the secondary question right, but IRT precision is a measure of precision under IRT, and ICC is a measure of reliability under CTT. The main difference is that CTT expresses reliability as a single value, while IRT expresses precision for different values of the underlying trait. This isn't specific to absolute agreement, though, so maybe I'm misunderstanding.</p>
<h2>References</h2>
<ul>
<li><a href="http://deepblue.lib.umich.edu/bitstream/handle/2027.42/67956/10.1177_001316445001000103.pdf?sequence=2&amp;isAllowed=y" rel="nofollow noreferrer">Coombs, C. H. (1950). The Concepts of Reliability and Homogeneity. Educational and Psychological Measurement, 10(1), 43-56.</a></li>
<li><a href="http://psych.colorado.edu/%7Ecarey/courses/psyc5112/readings/alpha_cronbach.pdf" rel="nofollow noreferrer">Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. psychometrika, 16(3), 297-334.</a></li>
</ul>

CC BY-SA 3.0
2015-05-08T00:00:34.937Z
I do not understand your vocabulary: "accuracy is to validity."  Are accuracy and validity synonyms? Are reliability and precision synonyms under CTT?
Joel W
CC BY-SA 3.0
2015-05-08T06:27:01.053Z
It's more of a parallel. Put shortly, there is no "precision" in CTT, only "reliability."
Christian Hummeluhr
CC BY-SA 3.0
2015-05-06T20:29:59.150Z
2015-05-06T20:29:59.150Z
2020-06-17T09:55:31.633Z
Christian Hummeluhr
<p>What methods or approaches exist or might be developed to measure precision of measurement in a classical test theory framework? <em><a href="http://www.apa.org/science/programs/testing/standards.aspx" rel="nofollow">Standards for Educational and Psychological Testing</a></em> (American Psychological Association, 2014) talks about precision of measurement in several places, but does not seem to provide any references.</p>

<p>Also, to what extent is precision of measurement in IRT theory similar to or different from absolute agreement in score (as opposed to inter-rater reliability or test reliability as measured by alpha) in CTT?</p>

<measurement><methodology><test><statistics><reliability>
How to measure precision in classical test theory framework?
Joel W
# How to measure precision in classical test theory framework?

(Asked by: username_0 on 2015-02-26)

What methods or approaches exist or might be developed to measure precision of measurement in a classical test theory framework? *[Standards for Educational and Psychological Testing](http://www.apa.org/science/programs/testing/standards.aspx)* (American Psychological Association, 2014) talks about precision of measurement in several places, but does not seem to provide any references.

Also, to what extent is precision of measurement in IRT theory similar to or different from absolute agreement in score (as opposed to inter-rater reliability or test reliability as measured by alpha) in CTT?

**Tags:** measurement, methodology, test, statistics, reliability


---

## 1 answers

### Answer by username_1 on 2015-05-06. Score: 3

# 'Precision' in classical test theory

Most accounts of classical test theory do not have a notion of precision as such, but occasionally, *reliability* may be called precision instead. The relationship is probably most concisely illustrated with the standard dartboards. This is also explained on the Wikipedia [Item Response Theory](https://en.wikipedia.org/wiki/Item_response_theory#Information) page, but as you can see, in CTT, precision is to reliability what accuracy is to validity.

![enter image description here](https://i.sstatic.net/54faT.png) ![Tuft U](https://i.sstatic.net/hIBe9.png)

([Wikipedia Reliability article](https://en.wikipedia.org/wiki/File:Reliability_and_validity.svg) juxtaposed with [a Tufts University guide](https://wikis.uit.tufts.edu/confluence/display/MCM582/2012S01).

## Origin of 'precision' in classical test theory

Cronbach (1951) suggested Coombs (1950) as the origin of the reliability/precision confusion.

> Coombs (6) offers the somewhat more satisfactory name "coefficient of precision" for this index which reports the absolute minimum error to be found if the same instrument is applied twice independently to the same subject. A coefficient of stability can be obtained by making the two observations with any desired interval between. A rigorous definition of the coefficient of precision, then, is that it is the limit of the coefficient of stability, as the time between testings becomes infinitesimal.

I'm not entirely sure if I'm interpreting the secondary question right, but IRT precision is a measure of precision under IRT, and ICC is a measure of reliability under CTT. The main difference is that CTT expresses reliability as a single value, while IRT expresses precision for different values of the underlying trait. This isn't specific to absolute agreement, though, so maybe I'm misunderstanding.

## References

* [Coombs, C. H. (1950). The Concepts of Reliability and Homogeneity. Educational and Psychological Measurement, 10(1), 43-56.](http://deepblue.lib.umich.edu/bitstream/handle/2027.42/67956/10.1177_001316445001000103.pdf?sequence=2&isAllowed=y)
* [Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. psychometrika, 16(3), 297-334.](http://psych.colorado.edu/%7Ecarey/courses/psyc5112/readings/alpha_cronbach.pdf)

> **Comments:**
> **username_0**: I do not understand your vocabulary: "accuracy is to validity."  Are accuracy and validity synonyms? Are reliability and precision synonyms under CTT?  
> **username_1**: It's more of a parallel. Put shortly, there is no "precision" in CTT, only "reliability."  

---

