<p>A common test of <a href="https://en.wikipedia.org/wiki/Memory_span" rel="nofollow noreferrer">memory span</a> is to display or utter a sequence of numbers and then request that a candidate being assessed repeat that sequence of numbers in reverse order. Typically, the candidate's score is the length of the longest sequence that he or she repeated correctly.</p>
<p>Thus, to each sequence produced by the candidate it is assigned a score in <span class="math-container">$\{0,1\}$</span>, where score <span class="math-container">$0$</span> denotes <em>incorrect</em> and score <span class="math-container">$1$</span> denotes <em>correct</em>. However, what if the candidate does produce a sequence that differs from the correct sequence merely due to, say, two numbers having been swapped? Does this sequence deserve a score of <span class="math-container">$0$</span>? In my most humble opinion, permuting the reverse of the given sequence should not be a deadly sin, though it should be penalized.</p>
<p>Is there any work on <strong>scoring sequences</strong> in which the scores are in <span class="math-container">$[0,1]$</span> rather than in <span class="math-container">$\{0,1\}$</span>?</p>
<p>I thought of using the <a href="https://en.wikipedia.org/wiki/Levenshtein_distance" rel="nofollow noreferrer">Levenshtein</a> and <a href="https://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance" rel="nofollow noreferrer">Damerau-Levenshtein</a> distances and <a href="https://scholar.google.com/scholar?hl=en&amp;as_sdt=0%2C5&amp;q=%22digit%20span%22%20%2B%20Levenshtein" rel="nofollow noreferrer">found several papers</a> on Google Scholar. However, since we are working with sequences of numbers rather than sequences of characters, perhaps other metrics would be more appropriate.  I am an utter neophyte, am not familiar with the jargon in the field of psychometrics and I am not convinced that number of citations is a great proxy for quality. Thus, I would rather ask the experts (or non-neophytes).  I am happy to start a bounty to motivate those who have invested time and effort in learning psychometrics to post an answer. An answer listing good papers would be excellent.</p>
<hr />
<p><strong>Related</strong></p>
<ul>
<li><a href="https://psychology.stackexchange.com/q/1670/26475">How to reliably measure working memory capacity?</a></li>
</ul>

CC BY-SA 4.0
2021-06-08T20:01:25.480Z
As I recall when I administered this test, subjects were given 2 chances at each length, and failing both not only scores 0, but also stops the test.  With a partial score, you would also need to decide stopping criteria that can be determined on the fly by the administrator.  The current method has the advantage of being very simple to administer and score.
Arnon Weinberg
CC BY-SA 4.0
2021-06-09T13:35:57.413Z
@ArnonWeinberg Thank you for your comment. If I know very little about psychometrics, I know even less about its history. I assume that some of these tests were developed when computers were much more expensive and much, much slower. Given that computing is now relatively cheap, I am interested in gathering as much data as possible, as discarding data is much easier than acquiring it.
Rodrigo de Azevedo
<measurement><memory><reference-request><working-memory><short-term-memory>
Non-binary scores for backward digit span (BDS)
Rodrigo de Azevedo
# Non-binary scores for backward digit span (BDS)

(Asked by: username_0 on 2021-06-08)

A common test of [memory span](https://en.wikipedia.org/wiki/Memory_span) is to display or utter a sequence of numbers and then request that a candidate being assessed repeat that sequence of numbers in reverse order. Typically, the candidate's score is the length of the longest sequence that he or she repeated correctly.

Thus, to each sequence produced by the candidate it is assigned a score in $\{0,1\}$, where score $0$ denotes *incorrect* and score $1$ denotes *correct*. However, what if the candidate does produce a sequence that differs from the correct sequence merely due to, say, two numbers having been swapped? Does this sequence deserve a score of $0$? In my most humble opinion, permuting the reverse of the given sequence should not be a deadly sin, though it should be penalized.

Is there any work on **scoring sequences** in which the scores are in $[0,1]$ rather than in $\{0,1\}$?

I thought of using the [Levenshtein](https://en.wikipedia.org/wiki/Levenshtein_distance) and [Damerau-Levenshtein](https://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance) distances and [found several papers](https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=%22digit%20span%22%20%2B%20Levenshtein) on Google Scholar. However, since we are working with sequences of numbers rather than sequences of characters, perhaps other metrics would be more appropriate. I am an utter neophyte, am not familiar with the jargon in the field of psychometrics and I am not convinced that number of citations is a great proxy for quality. Thus, I would rather ask the experts (or non-neophytes). I am happy to start a bounty to motivate those who have invested time and effort in learning psychometrics to post an answer. An answer listing good papers would be excellent.

---

**Related**

* [How to reliably measure working memory capacity?](https://psychology.stackexchange.com/q/1670/26475)

**Tags:** measurement, memory, reference-request, working-memory, short-term-memory

> **Comments:**
> **username_1**: As I recall when I administered this test, subjects were given 2 chances at each length, and failing both not only scores 0, but also stops the test.  With a partial score, you would also need to decide stopping criteria that can be determined on the fly by the administrator.  The current method has the advantage of being very simple to administer and score.  
> **username_0**: @ArnonWeinberg Thank you for your comment. If I know very little about psychometrics, I know even less about its history. I assume that some of these tests were developed when computers were much more expensive and much, much slower. Given that computing is now relatively cheap, I am interested in gathering as much data as possible, as discarding data is much easier than acquiring it.  

---

## 0 answers

