<p>The assessment of reading comprehension is not my main area of interest, but some of the many articles on the topic of assessment of reading comprehension may be reflevant. See <a href="http://scholar.google.com.au/scholar?hl=en&amp;q=measurement+of+reading+comprehension&amp;btnG=Search&amp;as_sdt=0%2C5&amp;as_ylo=&amp;as_vis=0" rel="noreferrer">"measurement of reading comprehension"</a> on Google Scholar (e.g., a couple of the pdfs of reviews <a href="https://www.ideals.illinois.edu/bitstream/handle/2142/17663/ctrstreadtechrepv01978i00091_opt.pdf?sequence=1" rel="noreferrer">here</a>, and <a href="http://www.psych.lancs.ac.uk/people/uploads/katecain20090609T123758.pdf" rel="noreferrer">here</a>).</p>

<p>This entry in the <a href="http://literacyencyclopedia.ca/pdfs/Assessment_of_Reading_Comprehension.pdf" rel="noreferrer">literacy encyclopedia on assessment of reading comprehension</a> might be a good starting place.
The article has a the following quote of a definition of reading comprehension:</p>

<blockquote>
  <p>The National Assessment of Educational Progress (NAEP) 2009 Reading Framework Committee defines reading   comprehension as ...“an
  active and complex process that involves understanding    written
  text, developing and interpreting meaning, and using meaning as
  appropriate to   type of text, purpose and situation (National Center
  for Educational Statistics, 2005, p.    2).</p>
</blockquote>

<h3>Here are a few general points:</h3>

<ul>
<li>Multiple choice testing has the benefit of being reliable given a sufficient number of items. Of course, questions could be raised over the breadth of what it measures (i.e., validity issues). </li>
<li>Familiarity with the content should increase reading comprehension. Thus, to build on your question, a philosophy student should comprehend more Kant, than a non-philosophy student holding general reading ability constant. A similar argument could be made where the reader has insider knowledge about what the author is trying to say.</li>
<li>Some texts are more difficult to comprehend in general, whether it be because the text is poorly written (e.g., grammar, spelling, stylistic issues) or because of the complexity of the material.</li>
</ul>

<h3>Implication of the Rasch model to questions of measurement:</h3>

<p>A basic model of responding to multiple choice questions based on Item Response Theory (IRT) is the <a href="http://en.wikipedia.org/wiki/Rasch_model" rel="noreferrer">Rasch model</a>. It basically says that the probability of an individual getting an item correct is a function of their ability and item difficulty. </p>

<p>So according to this model, increasing the difficulty of the reading comprehension items would affect all individuals roughly equally. As can be shown with IRT the reliability of measurement can be increased by setting the difficulty of items to a level that discriminates between individuals. I.e., if items are too difficult, everyone gets the item wrong, and we don't learn much; likewise, if the item is too easy, everyone gets the item right, and we don't learn much.
Relating this to your question, including poorly worded writing or really difficult subject matters would only increase difficulty, which would not directly affect measurement of individual differences.</p>

<p>However, the Rasch model is generally simplification of the real world. In general, I think it takes you a long way in understanding the factors driving test performance. However, in some cases you might have reading items which some individuals understand better than others and this factor is unrelated to ability (e.g., a reading that embodies cultural assumptions shared by only some individuals). It is for this reason that reading comprehension tests should probably avoid content where a subgroup is likely to possess substantially more domain specific knowledge on the topic.</p>

CC BY-SA 3.0
2012-01-21T03:56:42.897Z
Very good points. And yeah - it's probably true - most reading comprehension passages are small, so they don't involve complex psychological issues that you'd see in, say, a longer book.
InquilineKea
CC BY-SA 3.0
2012-01-21T03:11:54.923Z
2012-01-21T03:11:54.923Z
Jeromy Anglim
<p>Certainly, you have reading comprehension passages on the SAT, GRE, and LSAT. <strong>But beyond that point, is it really possible to measure reading comprehension through multiple-choice tests?</strong> There are multiple motivations for why an author may make argument X, and multiple interpretations of his exact argument, which may be affected by the immediate circumstances that made him write that argument at that particular point in time (and while we may know a bit about the society that he lived in, chances are that we'll probably never know who he intended the book to be written for - hell - some people probably intended to write a book to impress some potential lover [and hell, maybe some passages in the book are motivated to attract the sympathy of that lover] even though they pretended to write it for the greater good of society). </p>

<p>I mean, all you have to do is to look at courtrooms. There, people pretty much argue about what people (in a case with precedent) actually meant, and there's usually no concrete (and correct) answer. And people who study the life of a particular philosopher may have multiple interpretations of why philosopher X wrote passage Y, and what X really meant in writing passage Y. </p>

<p>Plus, it's oftentimes very interesting to read the writing of someone you know REALLY well because you know their life so well that you can dream up hypotheses of what prompted them to write passage X at time Y (and you also know his social network so you could imagine what audience he had in mind, and what audience he might have secretly hoped to impress without mentioning it). And perhaps because of this, reading comprehension is not a skill with a single scale, since people are better at understanding some people than they are at understanding others.</p>

<p>I mean, I guess the ultimate test of reading comprehension would be to test someone on one of Immanuel Kant's books, or on some recent postmodernist book (made artificially difficult to read/parse), and then ask him "well, what is Kant trying to get at"? But is that really a valid test? Or maybe one where you have to dissect the entire structure of a book where X is pretending to be Y pretending to be Y pretending to be X (of course, such books are rarely written, but theoretically possible). </p>

<p><strong>Alternatively, wouldn't someone with exceptional reading comprehension be better at reading crappy writing?</strong> (because they're probably more able to process multiple thought-streams at once - which includes possible fragmentary/jumbled ideas?) A common assumption in reading comprehension tests is that the writing is "perfect", but this is frequently not the case, and people (especially older people) can get attention lapses when writing</p>

CC BY-SA 3.0
2016-11-21T19:45:41.947Z
This is a really interesting question. Much research in reading is based on equivalencies with the levels of basic schooling (e.g. a 3rd grade reading level, or a 7th) with a "ceiling" of high school graduation (or 12th grade reading level) as the highest tracked level. It's not clear as to whether a 12th grade reading level is the highest level that meaningfully *matters* to most people or whether the idea of reading at a 15th grade level, or a 74th, is cognitively meaningless because a 12th grade reading level represents true and final mastery over the reading skills domain.
Robert Columbia
CC BY-SA 3.0
2012-02-28T14:04:06.093Z
Is this the question? Amongst individuals who are around or above the average level of verbal intelligence, are differences in verbal comprehension more explainable by a variable/construct known as 'psychological intuitiveness'?
Jason McPherson
CC BY-SA 3.0
2012-01-19T03:28:24.733Z
Interesting musings, but I think this might be better suited to the philosophy SE. This reads more like an epistemology question, but that's just my interpretation ;)
zergylord
CC BY-SA 3.0
2012-01-19T03:57:16.467Z
Heh, I don't think that the Philosophy SE would appreciate this type of question. :)

But yes - I'm interested in the cognitive factors behind reading comprehension, and whether or not there is a "ceiling effect" with respect to improvements in reading comprehension ability, and if this "ceiling effect" has to do with understanding psychology.
InquilineKea
CC BY-SA 3.0
2012-01-20T01:25:27.847Z
I think it's okay to have  provide musings as context for a question, but ultimately there should be a clear articulation of the question. I think you might improve this question by adding at the end perhaps in dot points the 2 or 3 explicit single-sentence questions that you have.
Jeromy Anglim
CC BY-SA 3.0
2012-01-20T02:40:29.243Z
Okay - I've bolded the 2 explicit questions that I have. I need to attend to my other questions now, but please comment if you'd like it improved any further.
InquilineKea
<cognitive-psychology><intelligence><measurement><reading>
Beyond a certain level, is reading comprehension affected more by psychological intuition than by verbal intelligence?
InquilineKea
# Beyond a certain level, is reading comprehension affected more by psychological intuition than by verbal intelligence?

(Asked by: username_0 on 2012-01-18)

Certainly, you have reading comprehension passages on the SAT, GRE, and LSAT. **But beyond that point, is it really possible to measure reading comprehension through multiple-choice tests?** There are multiple motivations for why an author may make argument X, and multiple interpretations of his exact argument, which may be affected by the immediate circumstances that made him write that argument at that particular point in time (and while we may know a bit about the society that he lived in, chances are that we'll probably never know who he intended the book to be written for - hell - some people probably intended to write a book to impress some potential lover [and hell, maybe some passages in the book are motivated to attract the sympathy of that lover] even though they pretended to write it for the greater good of society).

I mean, all you have to do is to look at courtrooms. There, people pretty much argue about what people (in a case with precedent) actually meant, and there's usually no concrete (and correct) answer. And people who study the life of a particular philosopher may have multiple interpretations of why philosopher X wrote passage Y, and what X really meant in writing passage Y.

Plus, it's oftentimes very interesting to read the writing of someone you know REALLY well because you know their life so well that you can dream up hypotheses of what prompted them to write passage X at time Y (and you also know his social network so you could imagine what audience he had in mind, and what audience he might have secretly hoped to impress without mentioning it). And perhaps because of this, reading comprehension is not a skill with a single scale, since people are better at understanding some people than they are at understanding others.

I mean, I guess the ultimate test of reading comprehension would be to test someone on one of Immanuel Kant's books, or on some recent postmodernist book (made artificially difficult to read/parse), and then ask him "well, what is Kant trying to get at"? But is that really a valid test? Or maybe one where you have to dissect the entire structure of a book where X is pretending to be Y pretending to be Y pretending to be X (of course, such books are rarely written, but theoretically possible).

**Alternatively, wouldn't someone with exceptional reading comprehension be better at reading crappy writing?** (because they're probably more able to process multiple thought-streams at once - which includes possible fragmentary/jumbled ideas?) A common assumption in reading comprehension tests is that the writing is "perfect", but this is frequently not the case, and people (especially older people) can get attention lapses when writing

**Tags:** cognitive-psychology, intelligence, measurement, reading

> **Comments:**
> **username_3**: Interesting musings, but I think this might be better suited to the philosophy SE. This reads more like an epistemology question, but that's just my interpretation ;)  
> **username_0**: Heh, I don't think that the Philosophy SE would appreciate this type of question. :)  But yes - I'm interested in the cognitive factors behind reading comprehension, and whether or not there is a "ceiling effect" with respect to improvements in reading comprehension ability, and if this "ceiling effect" has to do with understanding psychology.  
> **username_4**: I think it's okay to have  provide musings as context for a question, but ultimately there should be a clear articulation of the question. I think you might improve this question by adding at the end perhaps in dot points the 2 or 3 explicit single-sentence questions that you have.  
> **username_0**: Okay - I've bolded the 2 explicit questions that I have. I need to attend to my other questions now, but please comment if you'd like it improved any further.  
> **username_2**: Is this the question? Amongst individuals who are around or above the average level of verbal intelligence, are differences in verbal comprehension more explainable by a variable/construct known as 'psychological intuitiveness'?  
> **username_1**: This is a really interesting question. Much research in reading is based on equivalencies with the levels of basic schooling (e.g. a 3rd grade reading level, or a 7th) with a "ceiling" of high school graduation (or 12th grade reading level) as the highest tracked level. It's not clear as to whether a 12th grade reading level is the highest level that meaningfully *matters* to most people or whether the idea of reading at a 15th grade level, or a 74th, is cognitively meaningless because a 12th grade reading level represents true and final mastery over the reading skills domain.  

---

## 1 answers

### Answer by username_4 on 2012-01-21. Score: 5 | Accepted answer

The assessment of reading comprehension is not my main area of interest, but some of the many articles on the topic of assessment of reading comprehension may be reflevant. See ["measurement of reading comprehension"](http://scholar.google.com.au/scholar?hl=en&q=measurement+of+reading+comprehension&btnG=Search&as_sdt=0%2C5&as_ylo=&as_vis=0) on Google Scholar (e.g., a couple of the pdfs of reviews [here](https://www.ideals.illinois.edu/bitstream/handle/2142/17663/ctrstreadtechrepv01978i00091_opt.pdf?sequence=1), and [here](http://www.psych.lancs.ac.uk/people/uploads/katecain20090609T123758.pdf)).

This entry in the [literacy encyclopedia on assessment of reading comprehension](http://literacyencyclopedia.ca/pdfs/Assessment_of_Reading_Comprehension.pdf) might be a good starting place.
The article has a the following quote of a definition of reading comprehension:

> The National Assessment of Educational Progress (NAEP) 2009 Reading Framework Committee defines reading comprehension as ...“an
> active and complex process that involves understanding written
> text, developing and interpreting meaning, and using meaning as
> appropriate to type of text, purpose and situation (National Center
> for Educational Statistics, 2005, p. 2).

### Here are a few general points:

* Multiple choice testing has the benefit of being reliable given a sufficient number of items. Of course, questions could be raised over the breadth of what it measures (i.e., validity issues).
* Familiarity with the content should increase reading comprehension. Thus, to build on your question, a philosophy student should comprehend more Kant, than a non-philosophy student holding general reading ability constant. A similar argument could be made where the reader has insider knowledge about what the author is trying to say.
* Some texts are more difficult to comprehend in general, whether it be because the text is poorly written (e.g., grammar, spelling, stylistic issues) or because of the complexity of the material.

### Implication of the Rasch model to questions of measurement:

A basic model of responding to multiple choice questions based on Item Response Theory (IRT) is the [Rasch model](http://en.wikipedia.org/wiki/Rasch_model). It basically says that the probability of an individual getting an item correct is a function of their ability and item difficulty.

So according to this model, increasing the difficulty of the reading comprehension items would affect all individuals roughly equally. As can be shown with IRT the reliability of measurement can be increased by setting the difficulty of items to a level that discriminates between individuals. I.e., if items are too difficult, everyone gets the item wrong, and we don't learn much; likewise, if the item is too easy, everyone gets the item right, and we don't learn much.
Relating this to your question, including poorly worded writing or really difficult subject matters would only increase difficulty, which would not directly affect measurement of individual differences.

However, the Rasch model is generally simplification of the real world. In general, I think it takes you a long way in understanding the factors driving test performance. However, in some cases you might have reading items which some individuals understand better than others and this factor is unrelated to ability (e.g., a reading that embodies cultural assumptions shared by only some individuals). It is for this reason that reading comprehension tests should probably avoid content where a subgroup is likely to possess substantially more domain specific knowledge on the topic.

> **Comments:**
> **username_0**: Very good points. And yeah - it's probably true - most reading comprehension passages are small, so they don't involve complex psychological issues that you'd see in, say, a longer book.  

---

