<p>It sounds like you are asking about the functional relationship between time spent on a test item and the probability of answering that item correctly.</p>
<p>In general, the functional form of such relationships for a given individual is sigmoidal. I.e., if you imagine the x-axis is time spent, and the y axis is probability of answering the item correctly for a particular individual, then it will start at the guessing rate (e.g., typically 1 / k where k is the number of response options) and then after a certain period of time it will rise and then plateau.</p>
<p>You could average over individuals to represent a typical time-performance curve for a given item type.</p>
<p>For instance, vocabulary items (e.g., a target word and several response options, one of which has the same meaning) do not take very long to reach the rise and plateau. For the most part, people either know the meaning of the word or they do not know the word. A lot of trivia and general knowledge items would be similar.</p>
<p>In contrast, some items require a somewhat elaborate algorithm to be followed. For instance, some 3d rotation test items require comparison of lots of elements.</p>
<p>Some items are easy given enough time. Perhaps think about certain perceptual speed items, like finding all the &quot;a's&quot; in a grid of letters (e.g., think about Where's Waldo/Wally puzzles). These tests are usually structured as a timed test, in that the ability is really about the speed at which something can be done.</p>
<p>More generally, items can often solved with different strategies that vary in accuracy and in time required. For example, various arithmetic problems (e.g., 37 * 37) can be solved by some quick strategy (involving retrieval and mental manipulation), but spending longer and applying various checks will yield a more accurate answer.</p>
<p>Ultimately, there is a distinction between power and speed tests. Power test don't have a time limit (or the time limit is not meant to interfere with performance substantially), and on speed tests, the time limit is the defining factor. Of course, there are also tests that fall somewhere in between.</p>
<p>From what I know about the Raven's, I imagine having more time would help, but there would also be an upper bound. In some cases, people may think they have found the pattern and be wrong. In other cases, they just would not get it.</p>

CC BY-SA 4.0
2022-02-16T01:05:11.740Z
Hi Jeromy, good to see you around again. Can you please give some sources to your answer?
AliceD
CC BY-SA 4.0
2022-02-14T23:50:11.117Z
2022-02-14T23:50:11.117Z
Jeromy Anglim
<p>As time limits are relaxed on the Raven's the correlation between 'working memory' (and conventional IQ score), and raw performance drops drastically. That has been known for over a decade, and yet there are still very strict time limits on fluid IQ tests. Essentially, if you limit time, you force &quot;IQ&quot; to correlate with what can be juggled in short term memory. That is a primary objection to IQ, and a reason why high IQ societies have allowed truly untimed, &quot;Power&quot; tests.</p>
<p><a href="https://www.sciencedirect.com/science/article/pii/S0160289613000445" rel="nofollow noreferrer">https://www.sciencedirect.com/science/article/pii/S0160289613000445</a></p>

CC BY-SA 4.0
2024-09-26T12:43:26.597Z
2024-09-26T12:43:26.597Z
rickwalterz69
<p>I know untimed tests exist, but they're still under the constraint of testing people that have lives to get to. People have jobs, loved ones, classes to go to, lives to live, and as such even untimed tests aren't really untimed, since you'd have to be really vain to spend 16 hours on an &quot;untimed test with no rewards&quot; just to get a better score. So the question is, as a mind experiment, if you take the ceiling item of a Raven's test and take an average Joe, then lock them in a room with a bed and nutrition for a week, and say if they don't solve it, they're gonna die akin to a Saw scenario, will they solve it or will they face their doom?</p>
<p>I've found the following relevant research:</p>
<ol>
<li>Richard P. DeShon, David Chan, Daniel A. Weissbein, Verbal overshadowing effects on Raven's advanced progressive matrices: Evidence for multidimensional performance determinants, Intelligence, Volume 21, Issue 2, 1995,Pages 135-155, ISSN 0160-2896,</li>
<li>Bors, Douglas &amp; Vigneau, François. (2001). The effect of practice on Raven's Advanced Progressive Matrices. Learning and Individual Differences - LEARN INDIVID DIFFER. 13. 291-312. 10.1016/S1041-6080(03)00015-3.</li>
<li>Kunda, Maithilee. (2010). Taking a look (literally!) at the Raven's intelligence test: Two visual solution strategies. 1691-1696.</li>
</ol>

CC BY-SA 4.0
2022-02-14T17:24:23.397Z
That seems like cheating, since you'd run out of wrong guesses. They cannot know their guesses are wrong. They must arrive to the correct solution through autonomous reasoning, not trial-and-error. But the time given would still be essentially unlimited, as would be their motivation to perform.
AshFromPheonix
CC BY-SA 4.0
2022-02-14T17:30:48.617Z
In the SE help, on the [don't ask](https://psychology.stackexchange.com/help/dont-ask) page it says "You should only ask practical, answerable questions based on actual problems that you face" - what's the practical question you're asking/actual problem you seek to solve?
Bryan Krause
CC BY-SA 4.0
2022-02-14T17:37:41.170Z
I think you're misunderstanding that page. I'm asking a concrete answerable question, which doesn't presuppose a subjective lens such as ethics, which I'm asking for my own lack of comprehension and struggle with such a question, because the concerned field is psychology. The question is "are all matrix reasoning problems solve-able by anyone given enough time". Clearly culturally-loaded problems aren't solve-able, because they require empirical priors. But matrix reasoning problems have no such priors. Hence the question.
AshFromPheonix
CC BY-SA 4.0
2022-02-14T18:18:05.823Z
I agree with @BryanKrause that the thought experiment will likely attract opinion-based answers, so I would focus on a practical question instead.  A testable question is "How much does extra time improve performance on Raven's Matrices?"  Likely there are diminishing returns, and improvement with extra time is asymptotic to some value.
Arnon Weinberg
CC BY-SA 4.0
2022-02-14T18:31:49.357Z
@ArnonWeinberg That does actually seem to be what happens with practice effect, upon repeats the returns are diminishing.
AshFromPheonix
CC BY-SA 4.0
2022-02-14T19:02:01.993Z
Respectfully, no, I don't think I am misunderstanding it. I think your question needs context as to what goal you are trying to solve. You also need to avoid the [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem).
Bryan Krause
<cognitive-psychology><intelligence>
Given unlimited time can any Raven's Matrix Reasoning problem be solved by anyone?
AshFromPheonix
# Given unlimited time can any Raven's Matrix Reasoning problem be solved by anyone?

(Asked by: username_0 on 2022-02-14)

I know untimed tests exist, but they're still under the constraint of testing people that have lives to get to. People have jobs, loved ones, classes to go to, lives to live, and as such even untimed tests aren't really untimed, since you'd have to be really vain to spend 16 hours on an "untimed test with no rewards" just to get a better score. So the question is, as a mind experiment, if you take the ceiling item of a Raven's test and take an average Joe, then lock them in a room with a bed and nutrition for a week, and say if they don't solve it, they're gonna die akin to a Saw scenario, will they solve it or will they face their doom?

I've found the following relevant research:

1. Richard P. DeShon, David Chan, Daniel A. Weissbein, Verbal overshadowing effects on Raven's advanced progressive matrices: Evidence for multidimensional performance determinants, Intelligence, Volume 21, Issue 2, 1995,Pages 135-155, ISSN 0160-2896,
2. Bors, Douglas & Vigneau, François. (2001). The effect of practice on Raven's Advanced Progressive Matrices. Learning and Individual Differences - LEARN INDIVID DIFFER. 13. 291-312. 10.1016/S1041-6080(03)00015-3.
3. Kunda, Maithilee. (2010). Taking a look (literally!) at the Raven's intelligence test: Two visual solution strategies. 1691-1696.

**Tags:** cognitive-psychology, intelligence

> **Comments:**
> **username_0**: That seems like cheating, since you'd run out of wrong guesses. They cannot know their guesses are wrong. They must arrive to the correct solution through autonomous reasoning, not trial-and-error. But the time given would still be essentially unlimited, as would be their motivation to perform.  
> **username_1**: In the SE help, on the [don't ask](https://psychology.stackexchange.com/help/dont-ask) page it says "You should only ask practical, answerable questions based on actual problems that you face" - what's the practical question you're asking/actual problem you seek to solve?  
> **username_0**: I think you're misunderstanding that page. I'm asking a concrete answerable question, which doesn't presuppose a subjective lens such as ethics, which I'm asking for my own lack of comprehension and struggle with such a question, because the concerned field is psychology. The question is "are all matrix reasoning problems solve-able by anyone given enough time". Clearly culturally-loaded problems aren't solve-able, because they require empirical priors. But matrix reasoning problems have no such priors. Hence the question.  
> **username_2**: I agree with @BryanKrause that the thought experiment will likely attract opinion-based answers, so I would focus on a practical question instead.  A testable question is "How much does extra time improve performance on Raven's Matrices?"  Likely there are diminishing returns, and improvement with extra time is asymptotic to some value.  
> **username_0**: @ArnonWeinberg That does actually seem to be what happens with practice effect, upon repeats the returns are diminishing.  
> **username_1**: Respectfully, no, I don't think I am misunderstanding it. I think your question needs context as to what goal you are trying to solve. You also need to avoid the [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem).  

---

## 2 answers

### Answer by username_3 on 2022-02-14. Score: 1 | Accepted answer

It sounds like you are asking about the functional relationship between time spent on a test item and the probability of answering that item correctly.

In general, the functional form of such relationships for a given individual is sigmoidal. I.e., if you imagine the x-axis is time spent, and the y axis is probability of answering the item correctly for a particular individual, then it will start at the guessing rate (e.g., typically 1 / k where k is the number of response options) and then after a certain period of time it will rise and then plateau.

You could average over individuals to represent a typical time-performance curve for a given item type.

For instance, vocabulary items (e.g., a target word and several response options, one of which has the same meaning) do not take very long to reach the rise and plateau. For the most part, people either know the meaning of the word or they do not know the word. A lot of trivia and general knowledge items would be similar.

In contrast, some items require a somewhat elaborate algorithm to be followed. For instance, some 3d rotation test items require comparison of lots of elements.

Some items are easy given enough time. Perhaps think about certain perceptual speed items, like finding all the "a's" in a grid of letters (e.g., think about Where's Waldo/Wally puzzles). These tests are usually structured as a timed test, in that the ability is really about the speed at which something can be done.

More generally, items can often solved with different strategies that vary in accuracy and in time required. For example, various arithmetic problems (e.g., 37 \* 37) can be solved by some quick strategy (involving retrieval and mental manipulation), but spending longer and applying various checks will yield a more accurate answer.

Ultimately, there is a distinction between power and speed tests. Power test don't have a time limit (or the time limit is not meant to interfere with performance substantially), and on speed tests, the time limit is the defining factor. Of course, there are also tests that fall somewhere in between.

From what I know about the Raven's, I imagine having more time would help, but there would also be an upper bound. In some cases, people may think they have found the pattern and be wrong. In other cases, they just would not get it.

> **Comments:**
> **username_4**: Hi Jeromy, good to see you around again. Can you please give some sources to your answer?  

---

### Answer by username_5 on 2024-09-26. Score: 0

As time limits are relaxed on the Raven's the correlation between 'working memory' (and conventional IQ score), and raw performance drops drastically. That has been known for over a decade, and yet there are still very strict time limits on fluid IQ tests. Essentially, if you limit time, you force "IQ" to correlate with what can be juggled in short term memory. That is a primary objection to IQ, and a reason why high IQ societies have allowed truly untimed, "Power" tests.

<https://www.sciencedirect.com/science/article/pii/S0160289613000445>


---

