<p>So basically [for me] this question is whether there exists a learning model informed by the structure and usage of the Chinese writing.</p>
<p>The answer is yes, see for example <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5051716/" rel="nofollow noreferrer">Loach &amp; Wang (2016)</a>:</p>
<blockquote>
<p>The question we address is the optimum order in which Chinese characters should be learned.</p>
<p>There are two orders that make intuitive sense: in order of usage frequency, from high to low, and in order of network hierarchy, starting with primitives and building up compound characters using components that have already been learned. The first of these follows directly from the goal of the learner but the second merits further discussion. In general terms, the desirability of learning characters in hierarchal order follows from a broad principle of human cognition, that mastery of a complex system rests on mastery of the relevant features of its sub-components [14, 15]. This applies to Chinese characters if one assumes that it is productive to treat them as a complex system rather than as a set of unrelated symbols to be learned by rote. A number of experimental studies indicate that this assumption is valid. They show that orthographic awareness is of critical importance to skilled native readers and in learning to read by both Chinese children and second language learners [16–22]. These also show that orthographic awareness is present whether or not it is taught explicitly and, among learners, that the extent of the awareness is correlated with performance [23]. We consider learning characters in order of hierarchy to be desirable because we infer that a learning order that explicitly reflects orthographic principles is more likely to generate accurate and productive orthographic awareness in students.</p>
<p>There is, however, necessarily a tension between learning by usage frequency and learning by hierarchy, because frequency is only weakly correlated with character complexity. [...] Learning characters in order of frequency would therefore often mean learning characters before their components had been learned, whereas learning them in order of hierarchy would often mean learning rarer characters in advance of more common ones.</p>
<p>When devising a learning order one can choose either of the extremes, of frequency or hierarchy, or attempt to find a balance between them in which some common characters are learned in advance of their components. One previous approach that searched for such a balance was a network theory-based approach by Yan et al. [24]. They demonstrated that an algorithmically-optimized, balanced order can be substantially more efficient than one that follows frequency. [...]</p>
<p>The tension between frequency and hierarchy is a dominant consideration in determining the learning order but it is not the only one. Small-scale character-to-character patterns are also known to be important, especially for encouraging orthographic awareness [25–27]. Patterns can be chosen to emphasize the logic of character construction, by introducing components directly before their compounds, or to emphasize the functional role of components, by presenting their compounds in sets. These patterns are often found in human-curated orders, and especially in books on learning Chinese characters (for example, those of Heisig and Richardson [28, 29]). They embody sound educational principles, which can be understood in terms of Marton’s <a href="https://www.birmingham.ac.uk/research/activity/education/learning-study/variation-theory/index.aspx" rel="nofollow noreferrer">variation theory</a> [30–33]. [...]</p>
<p>Our algorithm is built on the fundamental assumption that hierarchal orders are the pedagogically desirable way to accumulate usage frequency and we search among this subset of orders for the one that is most efficient. The algorithm is implemented using the conceptual framework of network theory, within which we conceive the network of Chinese characters as a directed analytic graph [34]. The nodes in the graph represent characters and the edges represent the structural relationships between them. We devise a measure of node centrality that relates each character’s usage frequency to the effort required to learn it, and order the characters by this measure to provide a first approximation to the optimal learning order. We then sort this list into topological (hierarchal) order using an algorithm designed to minimally disturb the starting order. The algorithm can be applied to any scheduling task where nodes have intrinsic differences in importance and must be visited in topological order.</p>
</blockquote>
<p>Looking at the quantitative evaluation section of this paper though, the prior work of <a href="https://doi.org/10.1371/journal.pone.0069745" rel="nofollow noreferrer">Yan et. al</a> seems &quot;good enough&quot; though, and it might less work to implement. I don't know if either of that is suitable for your application or not, as it might require you to redesign it from the ground up. (Also, beware that I can't write in Chinese myself.)</p>

CC BY-SA 4.0
2018-08-21T16:29:14.093Z
Thanks! That's at least a part of the answer. I think that the question still stands though - given a list of characters and no way to change the order, how should I decide the ordering of the repetitions, as in my example described in the question?
d33tah
CC BY-SA 4.0
2018-08-22T17:48:49.413Z
You should not change the sequence of writing the character. It's the same with playing an instrument. You should not change figuring, halfway though, when learning a piano piece.
adamaero
CC BY-SA 4.0
2018-08-23T00:19:32.333Z
@d33tah: I'm not sure I understand your premises. Are you trying to write a completely stateless app? Is that why you can't/won't record what the user managed or didn't manage to remember?
got trolled too much this week
CC BY-SA 4.0
2018-08-23T06:37:13.897Z
@Fizz: in a way. It's stateless because it creates a worksheet that you're supposed to print it out. I came up with one compatible learning scheme, but feel it could be made more efficient.
d33tah
CC BY-SA 4.0
2018-08-21T01:19:53.653Z
2018-08-21T01:31:05.857Z
2020-06-17T09:55:31.633Z
got trolled too much this week
<p>Spaced repetition is based off the forgetting curve. Each review per day strengthens the memory:</p>

<p><a href="https://i.sstatic.net/JIDQD.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/JIDQD.png" alt="enter image description here"></a></p>

<p>Both Anki flashcards and SuperMemo also utilize the Leitner (box) system: 
<p><a href="https://commons.wikimedia.org/wiki/File:Leitner_system_animation.gif#/media/File:Leitner_system_animation.gif" rel="nofollow noreferrer"><img src="https://upload.wikimedia.org/wikipedia/commons/a/ae/Leitner_system_animation.gif" alt="Leitner system animation.gif"></a><br>(Animation by <a href="//commons.wikimedia.org/wiki/User:Zirguezi" rel="nofollow noreferrer" title="User:Zirguezi">Zirguezi</a> - CC0.)</p>

<p>Other freeware, such as Cram or Quizlet, do not have these enhanced features. They're more for the quick and dirty academic game. The material is only meant to linger for a semester or two. Then it essentially becomes lost in the abyss.</p>

<p>Yet, that's not quite the same breakdown for learning how to <em>write</em> Chinese characters correctly. Using the metaphor that the brain is a muscle: It flexes more when writing. See <a href="http://psyedu.org.cn/qguan/%E4%B8%AA%E4%BA%BA%E7%BD%91%E9%A1%B5%E4%BF%A1%E6%81%AF/%E5%8F%91%E8%A1%A8%E8%AE%BA%E6%96%87B-C/2012/B-4Guan%20Perfetti%20et%20al-2011.pdf" rel="nofollow noreferrer"><em>Writing Strengthens Orthography and Alphabetic-Coding Strengthens Phonology in Learning to Read Chinese</em></a>.  Writing is more active than reading. The more senses engaged the better. SuperMemo and Anki have algorithms based primarily from <em>seeing</em> the information. (Granted, Anki does have the feature where you can write directly on the application.)</p>

<p>At the end of the day there still is a hierarchy based on the user's "history" with the each item. (Also, easier material or material with less chunks will strengthen, therefore lengthen the time until subsequent review.) <strong>Memory has a natural atrophy. There is no avoiding this</strong> (aside from <a href="https://en.wikipedia.org/wiki/Hyperthymesia" rel="nofollow noreferrer">abnormal exceptions</a>). Think of normal brains as having a half-life, a natural rate of decay.</p>

<p>To answer you directly: There are only a few ways: brute repetition, spaced repetition, chunking and the Leitner system. Your app involves repetition with chunking. You could improve from there with different combinations of the current design with spaced repetition and the Leitner system. EDIT: You can approximate a rough spaced-repetition interval, but this will depend on if the user actually completes the worksheet in one sitting and how serious they keep up a consistent practice.</p>

<p>Furthermore, different users will have different obligations and habit forming tendencies. Having added functions, will improve user's experience. For example, a tag feature in Anki allows users to study only cards with the same tag. This is useful for, say, having an exam come up involving a large deck. I recommend sifting though <a href="https://apps.ankiweb.net/docs/manual.html" rel="nofollow noreferrer">Anki's user manual</a> for more ideas.</p>

<hr>

<p><em>Bonus</em></p>

<p>I actually was developing my own Android app based on using spaced repetition to remember larger bodies of information (opposed to simple flashcards). The spacing effect for memory is <strong>roughly triple the time</strong> (immediately again, 3 min, 9 min, 30 min, 90 min, etc). Needless to say, I put that project on the backburner. However, there is a similar <a href="https://play.google.com/store/apps/details?id=com.pianopracticeassistant.ppa" rel="nofollow noreferrer">app for memorizing music pieces</a>. (I am not involved, no kickbacks.)</p>

CC BY-SA 4.0
2018-08-22T16:37:55.597Z
Thanks - a lot of useful information here! I'm just a bit confused about how could I apply Leitner system in an offline environment - basically on an exercise set printed on paper. Any ideas here?
d33tah
CC BY-SA 4.0
2018-08-22T17:47:07.557Z
The animation displays the Leitner system, offline. If the user gets it correct, then it moves to the next box. https://en.wikipedia.org/wiki/Leitner_system
adamaero
CC BY-SA 4.0
2018-08-22T19:23:53.433Z
It's offline but **interactive** - I can't apply it to the program that generates a printable sheet unless I split it by character (or characters) and the user handles the box manually. What I'm looking for is an an approximate prediction on how the boxes would behave upfront, for an average user.
d33tah
CC BY-SA 4.0
2018-08-22T19:37:58.907Z
It can't be done upfront. It's based on feedback. Foresight has no place. I've never heard of non-causal learning software (https://en.wikipedia.org/wiki/Causal_system). It would be better to just scrap the Leitner system idea.
adamaero
CC BY-SA 4.0
2018-08-22T16:09:34.480Z
2018-08-23T13:39:49.420Z
2018-08-23T13:39:49.420Z
adamaero
<p>I recently wrote <a href="https://github.com/d33tah/strokes" rel="nofollow noreferrer">an open-source application</a> that lets you practice Chinese writing and the decisions I made about its inner workings were related to my theories of how human memory works. I'm not sure which facts specifically are relevant, so I'll explain it all in context (scroll down to the the "question" section if you don't believe it's relevant):</p>

<p>The goal of the application is to help user remember the writing of Chinese characters and their pronunciation. The assumption is that the user would input a set of Chinese characters, most of which are non-trivial to write with aid, let alone from memory. Moreover, there is the concept of stroke order which is useful to remember: each character is made of strokes which should be written in a specific shape, position and order. This makes it pretty complex. I heard that the way Chinese people cope with learning this is that they repeat writing of each character 1000 times, but I believe this is not the best use of time, given the number of characters.</p>

<p>The way my program works is that for each input character, user is taught each stroke four times - 1) alone, 2) with all strokes introduced so far, 3) in context and 4) in context but with no aid -- like in this example:</p>

<p><a href="https://i.sstatic.net/rB8Gb.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/rB8Gb.png" alt="example output of my program"></a></p>

<p>In this example, user is exposed to first four digits in Chinese and should fill in the highlighted strokes, as well as ones already introduced. In each fourth tile, user is expected to fill in all already introduced strokes with no aid, forcing them to focus on geometry.</p>

<p>As you can see, tiles are split into groups that have thicker borders. After introducing the first two characters, there is a special group of tiles at the end of second and beginning of third line: it is blank and contains a pronunciation hint. The user is expected to try to recall the character and write it in its entirety in those tiles.</p>

<h1>Question</h1>

<p>This is the part I have a question about: the goal of those specific tiles is to try to get the user to try to recall the characters hoping they eventually land in long-term memory, which should already not be in the working memory because it usually takes more than five minutes to write two characters. Then user is expected to practice next two characters and the drill repeats. Then, user is expected to recall all four characters introduced so far, as shown in the graphic above. This pattern repeats and for characters ABCDEFGH would look like following - hopefully you can deduce how it works:</p>

<pre><code>A
B
AB
C
D
CD
ABCD
E
F
EF
G
H
EFGH
ABCDEFGH
</code></pre>

<p>I showed it to a person interested in psychology and she immediately said: "this reminds me of spaced repetition"! I read about this and also found SuperMemo, and a program that implements it: Anki flashcards. </p>

<p>The problem is that both those programs are based on a formula that prognosticates if the user has already memorized a certain item; with that, they assign relative importance to review this item. </p>

<p>Is there another statistical model of human memory I could use for my application that doesn't have this requirement, but - for example - assumes an "average person" and a certain expected probability that this character will be at a given time remembered?</p>

CC BY-SA 4.0
2018-08-22T19:44:45.517Z
The current program described sounds good enough. Spaced repetition is based on feedback. If users are printing these worksheets, feedback with an algorithm can't happen. To see for yourself, try Anki: https://ankiweb.net/shared/decks/chinese (It's free for Android and desktop.) Cards that the user gets wrong are repeated in the same session (same day). Additionally, writing characters alone does not need to be memorized. It would be better to provide short quizzes involving sentences from the corpus. No man is an island; no character...should be isolated from real-world practice.
adamaero
<cognitive-psychology><learning><long-term-memory><writing>
Is there a non-interactive alternative to the concept of SuperMemo/spaced repetition?
d33tah
# Is there a non-interactive alternative to the concept of SuperMemo/spaced repetition?

(Asked by: username_0 on 2018-08-17)

I recently wrote an open-source application that lets you practice Chinese writing and the decisions I made about its inner workings were related to my theories of how human memory works. I'm not sure which facts specifically are relevant, so I'll explain it all in context (scroll down to the the "question" section if you don't believe it's relevant):
The goal of the application is to help user remember the writing of Chinese characters and their pronunciation. The assumption is that the user would input a set of Chinese characters, most of which are non-trivial to write with aid, let alone from memory. Moreover, there is the concept of stroke order which is useful to remember: each character is made of strokes which should be written in a specific shape, position and order. This makes it pretty complex. I heard that the way Chinese people cope with learning this is that they repeat writing of each character 1000 times, but I believe this is not the best use of time, given the number of characters.
The way my program works is that for each input character, user is taught each stroke four times - 1) alone, 2) with all strokes introduced so far, 3) in context and 4) in context but with no aid -- like in this example:

In this example, user is exposed to first four digits in Chinese and should fill in the highlighted strokes, as well as ones already introduced. In each fourth tile, user is expected to fill in all already introduced strokes with no aid, forcing them to focus on geometry.
As you can see, tiles are split into groups that have thicker borders. After introducing the first two characters, there is a special group of tiles at the end of second and beginning of third line: it is blank and contains a pronunciation hint. The user is expected to try to recall the character and write it in its entirety in those tiles.
Question
This is the part I have a question about: the goal of those specific tiles is to try to get the user to try to recall the characters hoping they eventually land in long-term memory, which should already not be in the working memory because it usually takes more than five minutes to write two characters. Then user is expected to practice next two characters and the drill repeats. Then, user is expected to recall all four characters introduced so far, as shown in the graphic above. This pattern repeats and for characters ABCDEFGH would look like following - hopefully you can deduce how it works:
A
B
AB
C
D
CD
ABCD
E
F
EF
G
H
EFGH
ABCDEFGH

I showed it to a person interested in psychology and she immediately said: "this reminds me of spaced repetition"! I read about this and also found SuperMemo, and a program that implements it: Anki flashcards. 
The problem is that both those programs are based on a formula that prognosticates if the user has already memorized a certain item; with that, they assign relative importance to review this item. 
Is there another statistical model of human memory I could use for my application that doesn't have this requirement, but - for example - assumes an "average person" and a certain expected probability that this character will be at a given time remembered?

**Tags:** cognitive-psychology, learning, long-term-memory, writing

> **Comments:**
> **username_1**: The current program described sounds good enough. Spaced repetition is based on feedback. If users are printing these worksheets, feedback with an algorithm can't happen. To see for yourself, try Anki: https://ankiweb.net/shared/decks/chinese (It's free for Android and desktop.) Cards that the user gets wrong are repeated in the same session (same day). Additionally, writing characters alone does not need to be memorized. It would be better to provide short quizzes involving sentences from the corpus. No man is an island; no character...should be isolated from real-world practice.  

---

## 2 answers

### Answer by username_1 on 2018-08-22. Score: 2 | Accepted answer

Spaced repetition is based off the forgetting curve. Each review per day strengthens the memory:

[![enter image description here](https://i.sstatic.net/JIDQD.png)](https://i.sstatic.net/JIDQD.png)

Both Anki flashcards and SuperMemo also utilize the Leitner (box) system:

[![Leitner system animation.gif](https://upload.wikimedia.org/wikipedia/commons/a/ae/Leitner_system_animation.gif)](https://commons.wikimedia.org/wiki/File:Leitner_system_animation.gif#/media/File:Leitner_system_animation.gif)  
(Animation by [Zirguezi](//commons.wikimedia.org/wiki/User:Zirguezi "User:Zirguezi") - CC0.)

Other freeware, such as Cram or Quizlet, do not have these enhanced features. They're more for the quick and dirty academic game. The material is only meant to linger for a semester or two. Then it essentially becomes lost in the abyss.

Yet, that's not quite the same breakdown for learning how to *write* Chinese characters correctly. Using the metaphor that the brain is a muscle: It flexes more when writing. See [*Writing Strengthens Orthography and Alphabetic-Coding Strengthens Phonology in Learning to Read Chinese*](http://psyedu.org.cn/qguan/%E4%B8%AA%E4%BA%BA%E7%BD%91%E9%A1%B5%E4%BF%A1%E6%81%AF/%E5%8F%91%E8%A1%A8%E8%AE%BA%E6%96%87B-C/2012/B-4Guan%20Perfetti%20et%20al-2011.pdf). Writing is more active than reading. The more senses engaged the better. SuperMemo and Anki have algorithms based primarily from *seeing* the information. (Granted, Anki does have the feature where you can write directly on the application.)

At the end of the day there still is a hierarchy based on the user's "history" with the each item. (Also, easier material or material with less chunks will strengthen, therefore lengthen the time until subsequent review.) **Memory has a natural atrophy. There is no avoiding this** (aside from [abnormal exceptions](https://en.wikipedia.org/wiki/Hyperthymesia)). Think of normal brains as having a half-life, a natural rate of decay.

To answer you directly: There are only a few ways: brute repetition, spaced repetition, chunking and the Leitner system. Your app involves repetition with chunking. You could improve from there with different combinations of the current design with spaced repetition and the Leitner system. EDIT: You can approximate a rough spaced-repetition interval, but this will depend on if the user actually completes the worksheet in one sitting and how serious they keep up a consistent practice.

Furthermore, different users will have different obligations and habit forming tendencies. Having added functions, will improve user's experience. For example, a tag feature in Anki allows users to study only cards with the same tag. This is useful for, say, having an exam come up involving a large deck. I recommend sifting though [Anki's user manual](https://apps.ankiweb.net/docs/manual.html) for more ideas.

---

*Bonus*

I actually was developing my own Android app based on using spaced repetition to remember larger bodies of information (opposed to simple flashcards). The spacing effect for memory is **roughly triple the time** (immediately again, 3 min, 9 min, 30 min, 90 min, etc). Needless to say, I put that project on the backburner. However, there is a similar [app for memorizing music pieces](https://play.google.com/store/apps/details?id=com.pianopracticeassistant.ppa). (I am not involved, no kickbacks.)

> **Comments:**
> **username_0**: Thanks - a lot of useful information here! I'm just a bit confused about how could I apply Leitner system in an offline environment - basically on an exercise set printed on paper. Any ideas here?  
> **username_1**: The animation displays the Leitner system, offline. If the user gets it correct, then it moves to the next box. https://en.wikipedia.org/wiki/Leitner_system  
> **username_0**: It's offline but **interactive** - I can't apply it to the program that generates a printable sheet unless I split it by character (or characters) and the user handles the box manually. What I'm looking for is an an approximate prediction on how the boxes would behave upfront, for an average user.  
> **username_1**: It can't be done upfront. It's based on feedback. Foresight has no place. I've never heard of non-causal learning software (https://en.wikipedia.org/wiki/Causal_system). It would be better to just scrap the Leitner system idea.  

---

### Answer by username_2 on 2018-08-21. Score: 2

So basically [for me] this question is whether there exists a learning model informed by the structure and usage of the Chinese writing.

The answer is yes, see for example [Loach & Wang (2016)](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5051716/):

> The question we address is the optimum order in which Chinese characters should be learned.
>
> There are two orders that make intuitive sense: in order of usage frequency, from high to low, and in order of network hierarchy, starting with primitives and building up compound characters using components that have already been learned. The first of these follows directly from the goal of the learner but the second merits further discussion. In general terms, the desirability of learning characters in hierarchal order follows from a broad principle of human cognition, that mastery of a complex system rests on mastery of the relevant features of its sub-components [14, 15]. This applies to Chinese characters if one assumes that it is productive to treat them as a complex system rather than as a set of unrelated symbols to be learned by rote. A number of experimental studies indicate that this assumption is valid. They show that orthographic awareness is of critical importance to skilled native readers and in learning to read by both Chinese children and second language learners [16–22]. These also show that orthographic awareness is present whether or not it is taught explicitly and, among learners, that the extent of the awareness is correlated with performance [23]. We consider learning characters in order of hierarchy to be desirable because we infer that a learning order that explicitly reflects orthographic principles is more likely to generate accurate and productive orthographic awareness in students.
>
> There is, however, necessarily a tension between learning by usage frequency and learning by hierarchy, because frequency is only weakly correlated with character complexity. [...] Learning characters in order of frequency would therefore often mean learning characters before their components had been learned, whereas learning them in order of hierarchy would often mean learning rarer characters in advance of more common ones.
>
> When devising a learning order one can choose either of the extremes, of frequency or hierarchy, or attempt to find a balance between them in which some common characters are learned in advance of their components. One previous approach that searched for such a balance was a network theory-based approach by Yan et al. [24]. They demonstrated that an algorithmically-optimized, balanced order can be substantially more efficient than one that follows frequency. [...]
>
> The tension between frequency and hierarchy is a dominant consideration in determining the learning order but it is not the only one. Small-scale character-to-character patterns are also known to be important, especially for encouraging orthographic awareness [25–27]. Patterns can be chosen to emphasize the logic of character construction, by introducing components directly before their compounds, or to emphasize the functional role of components, by presenting their compounds in sets. These patterns are often found in human-curated orders, and especially in books on learning Chinese characters (for example, those of Heisig and Richardson [28, 29]). They embody sound educational principles, which can be understood in terms of Marton’s [variation theory](https://www.birmingham.ac.uk/research/activity/education/learning-study/variation-theory/index.aspx) [30–33]. [...]
>
> Our algorithm is built on the fundamental assumption that hierarchal orders are the pedagogically desirable way to accumulate usage frequency and we search among this subset of orders for the one that is most efficient. The algorithm is implemented using the conceptual framework of network theory, within which we conceive the network of Chinese characters as a directed analytic graph [34]. The nodes in the graph represent characters and the edges represent the structural relationships between them. We devise a measure of node centrality that relates each character’s usage frequency to the effort required to learn it, and order the characters by this measure to provide a first approximation to the optimal learning order. We then sort this list into topological (hierarchal) order using an algorithm designed to minimally disturb the starting order. The algorithm can be applied to any scheduling task where nodes have intrinsic differences in importance and must be visited in topological order.

Looking at the quantitative evaluation section of this paper though, the prior work of [Yan et. al](https://doi.org/10.1371/journal.pone.0069745) seems "good enough" though, and it might less work to implement. I don't know if either of that is suitable for your application or not, as it might require you to redesign it from the ground up. (Also, beware that I can't write in Chinese myself.)

> **Comments:**
> **username_0**: Thanks! That's at least a part of the answer. I think that the question still stands though - given a list of characters and no way to change the order, how should I decide the ordering of the repetitions, as in my example described in the question?  
> **username_1**: You should not change the sequence of writing the character. It's the same with playing an instrument. You should not change figuring, halfway though, when learning a piano piece.  
> **username_2**: @username_0: I'm not sure I understand your premises. Are you trying to write a completely stateless app? Is that why you can't/won't record what the user managed or didn't manage to remember?  
> **username_0**: @Fizz: in a way. It's stateless because it creates a worksheet that you're supposed to print it out. I came up with one compatible learning scheme, but feel it could be made more efficient.  

---

