<p>Difficulty of an item could be operationalised in a variety of ways: probability of getting the item correct, amount of time required to complete, amount of resources required to complete, etc.</p>

<h3>Item response theory perspective</h3>

<p>That said, in the <a href="http://en.wikipedia.org/wiki/Item_response_theor">item response theory</a> literature item difficulty is typically synonymous with a parameter related to probability of getting an item correct. For example, in a basic three parameter logistic model, you might have a an equation like follows:</p>

<p>$$
p_i(\theta) = c_i + \frac{1 - c_i}{1 + \exp(-a_i(\theta - b_i))}
$$</p>

<p>where $\theta$ indexes individual ability, $c_i$ is a guessing parameter for item $i$, particularly relevant for multiple choice items, $a_i$ is the degree to which the item discriminates between individuals, and $b_i$ indexes item difficulty.</p>

<p>Of relevance to your question, this model asserts that the probability of getting an item correct is related to a number of factors including the ability of the individual and the difficulty of the item. </p>

<p>More complex models would permit individual by item interactions, where for example some individuals are better at math problems where as other are better at language problems.</p>

<h3>Cognitive psychology perspective</h3>

<p>Another perspective on item difficulty comes from cognitive psychology. We might alternatively call this feature of an item, its <em>complexity</em>. From such a perspective we could analyse the component processes of the task to develop a model of its complexity. From this perspective, <code>2 + 2 + 2</code> is more complex than <code>2 + 2</code> because it generally requires more operations to solve. E.g., an individual needs to first solve <code>2 + 2</code> as a subtask, store that value in working memory, and then <code>add 2</code> to the stored value. </p>

<p>However, I think a cognitive strategy learning perspective provides further insight. Ultimately, it may be the strategy that has a particular complexity rather than the item per se. And strategies are acquired with practice and training. For example, when a young child solves the problem <code>2 + 2</code>, they may have to apply the somewhat complex strategy of recognising this as an addition problem, counting fingers <code>1 and 2</code>, then counting two more fingers <code>1 and 2</code> and then counting all the finger <code>1,2,3,4</code>. Alternatively, when most adults see <code>2 + 2</code>, they are able to immediately and reliably retrieve the correct answer from memory. Or for example, if you are able to use a calculator, the difficulty of multiplication of large numbers is dramatically reduced.</p>

<p>There is a large literature on what is sometimes called algorithm-retrieval shift. I summarise some of this literature in my thesis (Anglim, 2011). To read more, you could check out Delaney et al (1988) or perhaps some of Robert Siegler's work (e.g., Lemaire and Siegler, 1995; he has a huge amount of research on strategies used in mathematical problems). I'd also encourage you to check out some of the many articles using the ACT-R framework (see below). The main point is that over time, complex tasks become simplified as individuals optimise knowledge, strategies, and task representations to the task domain. </p>

<p>From such a perspective, 2 + 2 is easier from 2 + 2 + 2 for a number of reasons. It is simpler in terms of the typical strategies that are used, but also the structure of typical task environments means that <code>2 + 2 + 2</code> will be less frequently encountered than 2 + 2. Thus, individuals are more likely to switch to a retrieval process on 2 + 2.</p>

<h3>Summary</h3>

<ul>
<li>complexity and difficulty are strategy specific</li>
<li>the environment influences task and strategy learning</li>
<li>and thus, the environment can influence experienced complexity and difficulty</li>
</ul>

<h3>References</h3>

<ul>
<li>Anglim, J. (2011). Strategies in skill acquisition: reconciling continuous models of the learning curve with abrupt strategy shifts. <a href="http://repository.unimelb.edu.au/researchfile272085.pdf">PDF</a></li>
<li>Delaney, P. F., Reder, L. M., Staszewski, J. J., &amp; Ritter, F. E. (1998).
The strategy-specic nature of improvement: The power law ap-
plies by strategy within task. Psychological Science, 9(1) 1-7.</li>
<li>Lemaire, P., &amp; Siegler, R. S. (1995). Four aspects of strategic change:
Contributions to children's learning of multiplication. Journal of
Experimental Psychology: General, 124 , 83-{97.</li>
<li>Robert Sigler: See website for many publications with free PDFs <a href="http://www.psy.cmu.edu/~siegler/">http://www.psy.cmu.edu/~siegler/</a> </li>
<li>ACT-R Literature on Mathematical Problem Solving: <a href="http://act-r.psy.cmu.edu/publications/index.php?subtopic=22">http://act-r.psy.cmu.edu/publications/index.php?subtopic=22</a></li>
</ul>

CC BY-SA 3.0
2012-08-25T07:48:01.600Z
2012-08-25T07:48:01.600Z
Jeromy Anglim
<p>When presenting someone with an arithmetic item, let's say 2+2, what makes it difficult? Is the item 2+2+2 only longer, or also more difficult? </p>

<p>Does the mental operation of addition make the difficulty, or also the amount of mental operations? Or should the amount only be seen something influencing the length it takes to solve the item, but not it's difficulty? </p>

<p>Are there any publications that address the idea of difficulty for psychometric items which address this question? </p>

CC BY-SA 3.0
2012-08-25T00:39:45.837Z
How do you measure difficulty? Time to complete? Accuracy over many trials?
mac389
<mathematical-ability>
What makes the difficulty of an arithmetic item? The operation or its length?
Adam
# What makes the difficulty of an arithmetic item? The operation or its length?

(Asked by: username_0 on 2012-08-24)

When presenting someone with an arithmetic item, let's say 2+2, what makes it difficult? Is the item 2+2+2 only longer, or also more difficult?

Does the mental operation of addition make the difficulty, or also the amount of mental operations? Or should the amount only be seen something influencing the length it takes to solve the item, but not it's difficulty?

Are there any publications that address the idea of difficulty for psychometric items which address this question?

**Tags:** mathematical-ability

> **Comments:**
> **username_1**: How do you measure difficulty? Time to complete? Accuracy over many trials?  

---

## 1 answers

### Answer by username_2 on 2012-08-25. Score: 6 | Accepted answer

Difficulty of an item could be operationalised in a variety of ways: probability of getting the item correct, amount of time required to complete, amount of resources required to complete, etc.

### Item response theory perspective

That said, in the [item response theory](http://en.wikipedia.org/wiki/Item_response_theor) literature item difficulty is typically synonymous with a parameter related to probability of getting an item correct. For example, in a basic three parameter logistic model, you might have a an equation like follows:

$$
p\_i(\theta) = c\_i + \frac{1 - c\_i}{1 + \exp(-a\_i(\theta - b\_i))}
$$

where $\theta$ indexes individual ability, $c\_i$ is a guessing parameter for item $i$, particularly relevant for multiple choice items, $a\_i$ is the degree to which the item discriminates between individuals, and $b\_i$ indexes item difficulty.

Of relevance to your question, this model asserts that the probability of getting an item correct is related to a number of factors including the ability of the individual and the difficulty of the item.

More complex models would permit individual by item interactions, where for example some individuals are better at math problems where as other are better at language problems.

### Cognitive psychology perspective

Another perspective on item difficulty comes from cognitive psychology. We might alternatively call this feature of an item, its *complexity*. From such a perspective we could analyse the component processes of the task to develop a model of its complexity. From this perspective, `2 + 2 + 2` is more complex than `2 + 2` because it generally requires more operations to solve. E.g., an individual needs to first solve `2 + 2` as a subtask, store that value in working memory, and then `add 2` to the stored value.

However, I think a cognitive strategy learning perspective provides further insight. Ultimately, it may be the strategy that has a particular complexity rather than the item per se. And strategies are acquired with practice and training. For example, when a young child solves the problem `2 + 2`, they may have to apply the somewhat complex strategy of recognising this as an addition problem, counting fingers `1 and 2`, then counting two more fingers `1 and 2` and then counting all the finger `1,2,3,4`. Alternatively, when most adults see `2 + 2`, they are able to immediately and reliably retrieve the correct answer from memory. Or for example, if you are able to use a calculator, the difficulty of multiplication of large numbers is dramatically reduced.

There is a large literature on what is sometimes called algorithm-retrieval shift. I summarise some of this literature in my thesis (Anglim, 2011). To read more, you could check out Delaney et al (1988) or perhaps some of Robert Siegler's work (e.g., Lemaire and Siegler, 1995; he has a huge amount of research on strategies used in mathematical problems). I'd also encourage you to check out some of the many articles using the ACT-R framework (see below). The main point is that over time, complex tasks become simplified as individuals optimise knowledge, strategies, and task representations to the task domain.

From such a perspective, 2 + 2 is easier from 2 + 2 + 2 for a number of reasons. It is simpler in terms of the typical strategies that are used, but also the structure of typical task environments means that `2 + 2 + 2` will be less frequently encountered than 2 + 2. Thus, individuals are more likely to switch to a retrieval process on 2 + 2.

### Summary

* complexity and difficulty are strategy specific
* the environment influences task and strategy learning
* and thus, the environment can influence experienced complexity and difficulty

### References

* Anglim, J. (2011). Strategies in skill acquisition: reconciling continuous models of the learning curve with abrupt strategy shifts. [PDF](http://repository.unimelb.edu.au/researchfile272085.pdf)
* Delaney, P. F., Reder, L. M., Staszewski, J. J., & Ritter, F. E. (1998).
  The strategy-specic nature of improvement: The power law ap-
  plies by strategy within task. Psychological Science, 9(1) 1-7.
* Lemaire, P., & Siegler, R. S. (1995). Four aspects of strategic change:
  Contributions to children's learning of multiplication. Journal of
  Experimental Psychology: General, 124 , 83-{97.
* Robert Sigler: See website for many publications with free PDFs <http://www.psy.cmu.edu/~siegler/>
* ACT-R Literature on Mathematical Problem Solving: <http://act-r.psy.cmu.edu/publications/index.php?subtopic=22>


---

