<h3>General points about practice effects</h3>

<p>Intelligence tests differ in how much they are subject to practice effects.</p>

<p>Practice effects can also be distinguished:</p>

<ul>
<li><strong>Time between taking the test</strong>: The shorter the timeframe the more likely you will see practice related improvement.</li>
<li><strong>General practice on similar tests and similar items versus practice on the same set of items.</strong> In general, practice effects will be greater where the items are identical between test sessions. Adaptive testing where items change each time should show fewer practice effects. </li>
<li><strong>Whether feedback is present</strong>. In general participants do not get feedback on what was or was not the right answer, but if they do receive such feedback, then practice effects will be much greater, especially for item specific transfer.</li>
</ul>

<h3>Test-retest variation</h3>

<p>Another general point is that intelligence tests have a level of reliability such that a score on a test is only an estimate of an individual's true standing on the latent trait being measured. So obviously, some variation between administrations would be expected purely based on the standard error of measurement.</p>

<p>That said, the general model of test-retest reliability assumes reasonably controlled testing conditions and a participant that is applying reasonable effort.
When these conditions are not met, you could see much more variation:</p>

<ul>
<li>If a test taker applies minimal effort or is distracted, they may get a much lower score than their true level.</li>
<li>If a test taker cheats in some way they may get a much higher score than their true level.</li>
</ul>

<p>Issues of administration are more likely in an online environment particularly where there is either (a) no consequences for not applying full effort, or (b) an incentive to cheat.</p>

<h3>Interpreting scores of someone who has taken a test multiple times</h3>

<p>In general, if you are trying to measure intelligence, you should administer the test once and that first measure would be the measure of their intelligence. </p>

<p>Test administrations that follow immediately afterwards would be contaminated by practice effects. Unless you have a very good understanding of how practice effects operate and how they can be controlled for, then it would be difficult to incorporate information from these subsequent administrations.</p>

<p>That said, if you had additional information that said that on the first administration the participant was not concentrating or fundamentally failed to understand the test, then you may wish to tentatively rely on the score from a different administration.</p>

<p>In addition, a pattern of scores that differ by three standard deviations (e.g., 100, 115, 145) may suggest either that the test has reliability issues or that something strange is happening for that particular individual that you want to look into.</p>

CC BY-SA 3.0
2015-06-30T05:46:51.957Z
If one gets 100 after 115 then Practice effects  do not affect much, right? but then 140 is a strong indication of Practice effects  since the online test is same. this is  what struck me.
Michael
CC BY-SA 3.0
2015-06-30T05:49:02.133Z
To evaluate practice effects, you'd want to examine changes in performance for a large number of individuals. When looking at an individual case, it may be difficult to tease out practice effects from random variation.
Jeromy Anglim
CC BY-SA 3.0
2015-06-30T06:06:10.133Z
Also, I dont get your point on concentration , if one does not concentrate/understand, should not we conclude that person has less IQ since he failed to  concentrate /comprehend the test? i am keen to know why concentration should considered, my perception is , intelligent people find problem-solving interesting and do well without much concentration as  problem solving is a natural ability.I think this is the ' public perception ' also.
Michael
CC BY-SA 3.0
2015-06-30T06:09:36.407Z
People may not concentrate for transient reasons that presumably could be readily fixed. So no. Low scores for lack of attention suggest a problem with test administration and don't really reflect an individual's latent intelligence. I'm not referring to deeper issues with attentional processes that we might associate with neurocognitive disorders.
Jeromy Anglim
CC BY-SA 3.0
2015-06-30T11:04:54.807Z
@what, your cases are quite normal cause for lack of concentration, But consider a person who  does not like test. Does this  unwilling-to-take-test tendency / characteristic say anything particular about the intelligence of that person? Is there anything in literature?
Michael
CC BY-SA 3.0
2015-06-30T05:21:03.813Z
2015-06-30T05:47:28.150Z
2015-06-30T05:47:28.150Z
Jeromy Anglim
<p><strong>Specific context:</strong> A test subject took the same Raven's Standard Progressive Matrices IQ test three times and first scored 115, then 107, and finally 143. The same test was taken three times within 3 and a half hours. Since all three scores are different and appear random, how can one  asses these results? i.e. Assuming the individual did not cheat, what can be said about the intelligence of  test subject?   </p>

<p>More generally:</p>

<p><strong>How should one interpret intelligence test scores for an individual who has taken the same form of the same test multiple times in quick succession?</strong></p>

<measurement><experimental-psychology><intelligence><performance><iq>
How to interpret intelligence test scores for an individual who has taken a test multiple times in quick succession?
Michael
# How to interpret intelligence test scores for an individual who has taken a test multiple times in quick succession?

(Asked by: username_0 on 2015-06-29)

**Specific context:** A test subject took the same Raven's Standard Progressive Matrices IQ test three times and first scored 115, then 107, and finally 143. The same test was taken three times within 3 and a half hours. Since all three scores are different and appear random, how can one asses these results? i.e. Assuming the individual did not cheat, what can be said about the intelligence of test subject?

More generally:

**How should one interpret intelligence test scores for an individual who has taken the same form of the same test multiple times in quick succession?**

**Tags:** measurement, experimental-psychology, intelligence, performance, iq


---

## 1 answers

### Answer by username_1 on 2015-06-30. Score: 4 | Accepted answer

### General points about practice effects

Intelligence tests differ in how much they are subject to practice effects.

Practice effects can also be distinguished:

* **Time between taking the test**: The shorter the timeframe the more likely you will see practice related improvement.
* **General practice on similar tests and similar items versus practice on the same set of items.** In general, practice effects will be greater where the items are identical between test sessions. Adaptive testing where items change each time should show fewer practice effects.
* **Whether feedback is present**. In general participants do not get feedback on what was or was not the right answer, but if they do receive such feedback, then practice effects will be much greater, especially for item specific transfer.

### Test-retest variation

Another general point is that intelligence tests have a level of reliability such that a score on a test is only an estimate of an individual's true standing on the latent trait being measured. So obviously, some variation between administrations would be expected purely based on the standard error of measurement.

That said, the general model of test-retest reliability assumes reasonably controlled testing conditions and a participant that is applying reasonable effort.
When these conditions are not met, you could see much more variation:

* If a test taker applies minimal effort or is distracted, they may get a much lower score than their true level.
* If a test taker cheats in some way they may get a much higher score than their true level.

Issues of administration are more likely in an online environment particularly where there is either (a) no consequences for not applying full effort, or (b) an incentive to cheat.

### Interpreting scores of someone who has taken a test multiple times

In general, if you are trying to measure intelligence, you should administer the test once and that first measure would be the measure of their intelligence.

Test administrations that follow immediately afterwards would be contaminated by practice effects. Unless you have a very good understanding of how practice effects operate and how they can be controlled for, then it would be difficult to incorporate information from these subsequent administrations.

That said, if you had additional information that said that on the first administration the participant was not concentrating or fundamentally failed to understand the test, then you may wish to tentatively rely on the score from a different administration.

In addition, a pattern of scores that differ by three standard deviations (e.g., 100, 115, 145) may suggest either that the test has reliability issues or that something strange is happening for that particular individual that you want to look into.

> **Comments:**
> **username_0**: If one gets 100 after 115 then Practice effects  do not affect much, right? but then 140 is a strong indication of Practice effects  since the online test is same. this is  what struck me.  
> **username_1**: To evaluate practice effects, you'd want to examine changes in performance for a large number of individuals. When looking at an individual case, it may be difficult to tease out practice effects from random variation.  
> **username_0**: Also, I dont get your point on concentration , if one does not concentrate/understand, should not we conclude that person has less IQ since he failed to  concentrate /comprehend the test? i am keen to know why concentration should considered, my perception is , intelligent people find problem-solving interesting and do well without much concentration as  problem solving is a natural ability.I think this is the ' public perception ' also.  
> **username_1**: People may not concentrate for transient reasons that presumably could be readily fixed. So no. Low scores for lack of attention suggest a problem with test administration and don't really reflect an individual's latent intelligence. I'm not referring to deeper issues with attentional processes that we might associate with neurocognitive disorders.  
> **username_0**: @what, your cases are quite normal cause for lack of concentration, But consider a person who  does not like test. Does this  unwilling-to-take-test tendency / characteristic say anything particular about the intelligence of that person? Is there anything in literature?  

---

