<blockquote>
  <p>So I guess the question is whether such a test can be constructed at
  all. In other words, is there a category of problems that require some
  minimal intelligence level to solve them no matter how long one might
  try. Like the problem that could be solved in a minute by a smarter
  person but is impenetrable to you no matter how many years you spend
  thinking about it?</p>
</blockquote>

<p>If that is your question, the answer is no. No problems like this are currently known to exist, and there is reason to believe that such problems will never be found.</p>

<p>Intelligence in not like "this person can do A" and "this person can do A and B", but rather more like "this person can do X with medium reliability" and "this person can do X with high readability".</p>

CC BY-SA 3.0
2013-09-02T13:06:09.613Z
This is exactly what I suspected. After thinking about it for a while I came to conclusion that any problem can be solved by trial and error given enough time and scratch paper. So it looks like IQ is all about efficiency and reliability.
Andrew Butenko
CC BY-SA 3.0
2013-08-30T20:53:16.767Z
2013-08-30T20:53:16.767Z
John Henry
<p>Actually, standard IQ tests, such as Raven's matrices, tend to assess intelligence better if they are not timed. In <a href="http://www.sciencedirect.com/science/article/pii/0191886989901293" rel="nofollow">this paper</a> by Philip Vernon (1988) it was found that the g-factor extracted slightly more variance for the same test if the test was not timed than if it had a time limit. This means if you ask yourself: "What is this test measuring?", you can answer: "Intelligence" with more confidence if it was administered in an untimed setting. </p>

<p>IQ tests measure a host of other things, such as how well you are feeling on the day of the test, whether you are nervous when it comes to test-taking, how aquainted you are with this type of questions, etc. Giving people more time can attenuate some of these factors a bit. Timed variants are more prevalent for reasons of convenience, and also because it's easier to standardize the test situation.</p>

<p><a href="http://www.sciencedirect.com/science/article/pii/S0160289605800121" rel="nofollow">It has been argued</a> that there is little difference between timed and untimed tests because intelligence closely corresponds to working memory capacity (Kyllonen &amp; Christal, 1990):</p>

<blockquote>
  <p>Confirmatory factor analysis yielded consistently high estimates of
  the correlation between working-memory capacity and reasoning ability
  factors (r = .80 to .90). We also found differentiation between the
  two factors: Reasoning correlated comparatively highly with general
  knowledge; working-memory capacity correlated comparatively highly
  with processing speed.</p>
</blockquote>

<p>Some IQ tests even have subtests which measure this capacity directly: repeating a row of numbers in the same order or backwards. Giving you additional time will not help you reproduce the row of numbers, if it was initially too long for you to remember. </p>

<p>From the high correlation between working memory capacity and processing speed, it was concluded that IQ tests largely measure the ability to vary features of multiple items at the same time, i.e. to manipulate multiple objects in working memory. Crucially, during administration an IQ test such as Raven's matrices, you are not supposed to use pencil and paper or other cultural crutches: you work on the items '<a href="http://psycnet.apa.org/psycinfo/1999-11137-004" rel="nofollow">online</a>' in your head (Engle et al., 1999). The more objects you can hold 'online', the better you will do. <a href="http://www.sciencedirect.com/science/article/pii/S0160289608000044" rel="nofollow">Recent research</a> (Colom et al, 2008) goes even further to say that processing speed is not a good predictor on its own, once working memory capacity is removed:</p>

<blockquote>
  <p>The findings are consistent with the view that simple short-term
  storage largely accounts for the relationship between working memory
  and intelligence. Mental speed, updating, and the control of attention
  are not consistently related to working memory, and they are not
  genuinely associated with intelligence once the short-term storage
  component is removed.</p>
</blockquote>

<p>And of course, as always with measures of individual differences, it is important to keep in mind that IQ doesn't measure how many questions you completed - it measures how you rank in a group of peers which has done the same test under the same conditions.</p>

<p>References:</p>

<ul>
<li>Vernon, P. A. (1989). The generality of&lt; i> g. Personality and
Individual Differences, 10(7), 803-804.</li>
<li>Kyllonen, P. C., &amp; Christal, R. E. (1990). Reasoning ability is
(little more than) working-memory capacity?!. Intelligence, 14(4),
389-433.</li>
<li>Engle, R. W., Tuholski, S. W., Laughlin, J. E., &amp; Conway, A. R.
(1999). Working memory, short-term memory, and general fluid
intelligence: a latent-variable approach. Journal of Experimental
Psychology: General, 128(3), 309.</li>
<li>Colom, R., Abad, F. J., Quiroga, M. Á., Shih, P. C., &amp;
Flores-Mendoza, C. (2008). Working memory and intelligence are highly
related constructs, but why?. Intelligence, 36(6), 584-606.</li>
</ul>

CC BY-SA 3.0
2013-08-29T11:09:10.850Z
2013-09-04T12:27:02.333Z
2013-09-04T12:27:02.333Z
Ana
<p>Being able to measure intelligence (or the problem-solving aspect of it) quantitatively could be indispensable in fields like cognitive science or artificial intelligence. Obviously, IQ tests aren't culture neutral (questions are stated in an informal language, and often rely on prior knowledge). Ideally I am looking for these characteristics:</p>

<ol>
<li>measure problem solving ability</li>
<li>be culture neutral</li>
<li>be of <strong>unrestricted duration</strong></li>
</ol>

<p><a href="http://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices" rel="nofollow">Raven's Progressive Matrices</a> look almost perfect for the job (variations can be developed for testing animals or even computer programs) but they, like EVERY intelligence test i know of, require test to last not more than a specific amount time, thus, failing to satisfy the third requirement. So I guess the question is whether such a test can be constructed at all. In other words, is there a category of problems that require some minimal intelligence level to solve them no matter how long one might try. Like the problem that could be solved in a minute by a smarter person but is impenetrable to you no matter how many years you spend thinking about it? </p>

CC BY-SA 3.0
2013-08-29T14:41:06.833Z
Interesting question, welcome to cogsci!  Curious about why the unrestricted duration requirement?  Knowing a little context might help you get more relevant answers.
Krysta
<intelligence><problem-solving><artificial-intelligence>
Duration-independent intelligence tests
Andrew Butenko
# Duration-independent intelligence tests

(Asked by: username_0 on 2013-08-29)

Being able to measure intelligence (or the problem-solving aspect of it) quantitatively could be indispensable in fields like cognitive science or artificial intelligence. Obviously, IQ tests aren't culture neutral (questions are stated in an informal language, and often rely on prior knowledge). Ideally I am looking for these characteristics:

1. measure problem solving ability
2. be culture neutral
3. be of **unrestricted duration**

[Raven's Progressive Matrices](http://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices) look almost perfect for the job (variations can be developed for testing animals or even computer programs) but they, like EVERY intelligence test i know of, require test to last not more than a specific amount time, thus, failing to satisfy the third requirement. So I guess the question is whether such a test can be constructed at all. In other words, is there a category of problems that require some minimal intelligence level to solve them no matter how long one might try. Like the problem that could be solved in a minute by a smarter person but is impenetrable to you no matter how many years you spend thinking about it?

**Tags:** intelligence, problem-solving, artificial-intelligence

> **Comments:**
> **username_1**: Interesting question, welcome to cogsci!  Curious about why the unrestricted duration requirement?  Knowing a little context might help you get more relevant answers.  

---

## 2 answers

### Answer by username_2 on 2013-08-30. Score: 1 | Accepted answer

> So I guess the question is whether such a test can be constructed at
> all. In other words, is there a category of problems that require some
> minimal intelligence level to solve them no matter how long one might
> try. Like the problem that could be solved in a minute by a smarter
> person but is impenetrable to you no matter how many years you spend
> thinking about it?

If that is your question, the answer is no. No problems like this are currently known to exist, and there is reason to believe that such problems will never be found.

Intelligence in not like "this person can do A" and "this person can do A and B", but rather more like "this person can do X with medium reliability" and "this person can do X with high readability".

> **Comments:**
> **username_0**: This is exactly what I suspected. After thinking about it for a while I came to conclusion that any problem can be solved by trial and error given enough time and scratch paper. So it looks like IQ is all about efficiency and reliability.  

---

### Answer by username_3 on 2013-08-29. Score: 8

Actually, standard IQ tests, such as Raven's matrices, tend to assess intelligence better if they are not timed. In [this paper](http://www.sciencedirect.com/science/article/pii/0191886989901293) by Philip Vernon (1988) it was found that the g-factor extracted slightly more variance for the same test if the test was not timed than if it had a time limit. This means if you ask yourself: "What is this test measuring?", you can answer: "Intelligence" with more confidence if it was administered in an untimed setting.

IQ tests measure a host of other things, such as how well you are feeling on the day of the test, whether you are nervous when it comes to test-taking, how aquainted you are with this type of questions, etc. Giving people more time can attenuate some of these factors a bit. Timed variants are more prevalent for reasons of convenience, and also because it's easier to standardize the test situation.

[It has been argued](http://www.sciencedirect.com/science/article/pii/S0160289605800121) that there is little difference between timed and untimed tests because intelligence closely corresponds to working memory capacity (Kyllonen & Christal, 1990):

> Confirmatory factor analysis yielded consistently high estimates of
> the correlation between working-memory capacity and reasoning ability
> factors (r = .80 to .90). We also found differentiation between the
> two factors: Reasoning correlated comparatively highly with general
> knowledge; working-memory capacity correlated comparatively highly
> with processing speed.

Some IQ tests even have subtests which measure this capacity directly: repeating a row of numbers in the same order or backwards. Giving you additional time will not help you reproduce the row of numbers, if it was initially too long for you to remember.

From the high correlation between working memory capacity and processing speed, it was concluded that IQ tests largely measure the ability to vary features of multiple items at the same time, i.e. to manipulate multiple objects in working memory. Crucially, during administration an IQ test such as Raven's matrices, you are not supposed to use pencil and paper or other cultural crutches: you work on the items '[online](http://psycnet.apa.org/psycinfo/1999-11137-004)' in your head (Engle et al., 1999). The more objects you can hold 'online', the better you will do. [Recent research](http://www.sciencedirect.com/science/article/pii/S0160289608000044) (Colom et al, 2008) goes even further to say that processing speed is not a good predictor on its own, once working memory capacity is removed:

> The findings are consistent with the view that simple short-term
> storage largely accounts for the relationship between working memory
> and intelligence. Mental speed, updating, and the control of attention
> are not consistently related to working memory, and they are not
> genuinely associated with intelligence once the short-term storage
> component is removed.

And of course, as always with measures of individual differences, it is important to keep in mind that IQ doesn't measure how many questions you completed - it measures how you rank in a group of peers which has done the same test under the same conditions.

References:

* Vernon, P. A. (1989). The generality of< i> g. Personality and
  Individual Differences, 10(7), 803-804.
* Kyllonen, P. C., & Christal, R. E. (1990). Reasoning ability is
  (little more than) working-memory capacity?!. Intelligence, 14(4),
  389-433.
* Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R.
  (1999). Working memory, short-term memory, and general fluid
  intelligence: a latent-variable approach. Journal of Experimental
  Psychology: General, 128(3), 309.
* Colom, R., Abad, F. J., Quiroga, M. Á., Shih, P. C., &
  Flores-Mendoza, C. (2008). Working memory and intelligence are highly
  related constructs, but why?. Intelligence, 36(6), 584-606.


---

