<p>To answer your second question:</p>

<p>From a mathematical perspective, it will depend on the texts that are sampled to compile any list and the variances associated with those samples. Languages evolve and there is no true character frequency, only an approximation. Spoken Chinese will have a different character frequency to written Chinese because the spoken language has a high entropy. It is likely that the HSK test had a panel of experts select characters based on what they deemed to be suitable, whereas the list you referenced was mass sampled by some kind of web crawler.</p>

<p>It is inconceivable that every single electronic Chinese character on the internet has been sampled, and if it was that would be very biased as that would ignore all of the written works not uploaded to the internet. Depending on the format of the inputs there will be a huge difference, if lots of text was taken from 微薄, then the character 哈 might rank as common, purely from comments that are like "hahahaha" for example. If classical Chinese was sampled heavily then characters like 曰 might feature much more frequently than 说.</p>

<p>Some of the HSK 5 and 6 words are very political and business oriented, but there are not many scientific words.</p>

CC BY-SA 4.0
2018-08-01T17:48:26.583Z
It's mostly made of formal written modern Chinese, as you can see here: http://lingua.mtsu.edu/chinese-computing/old-version1/docs/index.html . You have a nice point of view, nonetheless.
Enrico Brasil
CC BY-SA 4.0
2018-07-30T14:25:53.760Z
2018-07-30T14:32:01.597Z
2018-07-30T14:32:01.597Z
Benedict W. J. Irwin
<p>To give some partial information regarding your questions, Olle Linge at Hacking Chinese has a recent (July 2020) article <a href="https://www.hackingchinese.com/what-important-words-are-missing-from-hsk/" rel="nofollow noreferrer">What important words are missing from HSK?</a>. One thing he mentions is that:</p>
<blockquote>
<p>It should be clear that HSK is not meant to be a representation of the
most commonly used Chinese words. This is very obvious in the lower
levels, where words like “train station” and “bus” are part of HSK1,
which has only 150 words in total. Those words are nowhere near the
top 150 words in Chinese in general, but they are of course important
for foreigners visiting and travelling in China, which probably is why
they are included.</p>
<p>Overall, I think the lower levels of HSK match the needs of foreign
students quite well.</p>
</blockquote>
<p>Thus, one possibility is that the creators wanted to make sure that foreigners learned what they needed to get around in China when visiting/studying there. This would bias the most frequently used words away from more general frequency lists.</p>
<p>This being said, Olle also came up with a list of words from a frequency list gleaned from movie and TV subtitles (the SUBTLEX-CH corpus [Cai and Brysbaert, 2010]) that either do not appear in HSK or are delayed (ie. they are in HSK but appear in a higher HSK level than expected, based on their frequency). Based on this analysis, Olle found that these types of words seemed to be missing from the HSK lists:</p>
<ul>
<li>Many single-character words are missing</li>
<li>Names of places and countries are missing</li>
<li>Regional variants are missing</li>
<li>Profanity is missing entirely</li>
<li>Foreign things are mostly missing</li>
<li>Particles in informal language are missing</li>
</ul>
<p>Seeing what is missing from HSK could help give some insight into the decision making process behind making the lists. Olle goes into more detail about the types of words missing in his article, if you want to know more.</p>
<p>Also, Olle did the same analysis for TOCFL (though with a different frequency list): <a href="https://www.hackingchinese.com/what-important-words-are-missing-from-tocfl/" rel="nofollow noreferrer">What important words are missing or delayed in TOCFL?</a></p>
<p><strong>An important practical note:</strong> if you want to learn the words that are &quot;missing&quot; from the HSK (or TOCFL) lists, Olle has generated the word lists and placed them at the end of the article so that people can use them freely for their own study.</p>
<p>Frequency List Reference:</p>
<p>Cai, Q., &amp; Brysbaert, M. (2010). SUBTLEX-CH: Chinese word and character frequencies based on film subtitles. PloS one, 5(6), e10729.</p>

CC BY-SA 4.0
2020-08-20T17:06:25.027Z
Some extra info: I chose to do the analysis you refer to based on movie subtitles because that's closer to what foreigners want to talk about. I also did (but never published) an analysis comparing HSK to Junda's frequency list. That's not very interesting, because it's basically a list of formal or written Chinese that obviously is very common in Chinese, but not suitable for beginners. This is not terribly interesting, so I focused on the SUBTLEX-CH corpus, which turned out to be quite interesting!
Olle Linge
CC BY-SA 4.0
2020-08-19T18:02:23.060Z
2020-08-19T18:08:09.020Z
2020-08-19T18:08:09.020Z
2ndQuantized
<p>I have been wondering how come the HSK list be so different from the list of the most used Chinese characters.</p>
<p>The only list I know about character frequency is <a href="http://lingua.mtsu.edu/chinese-computing/statistics/char/list.php?Which=MO" rel="nofollow noreferrer">Jun Da's list</a>. On the first 2635 most common characters of the list (I picked 2635, because it's the total number of the HSK characters), there are exactly 305 characters that are not listed on any HSK.</p>
<p>On the rest of Jun Da's list (a total of 7000 characters) there are another 304 characters that are on the HSK list, but are way way farther than the most used characters (the last two rank at #4527 and #4771).</p>
<p><strong>Questions:</strong></p>
<ol>
<li>What are the criteria for the character choice on HSK?</li>
<li>Why is the HSK list so different from the most used characters list?</li>
</ol>
<p>Additional comments are very welcome.</p>

CC BY-SA 4.0
2018-07-09T08:55:02.363Z
What do you mean by *"I picked 2635, because it's the total number of the HSK characters"*? I googled by "hsk number of characters 2635" and this post showed at the top of the result, so I'm sure the figure is not authoritative at all.
Blaszard
CC BY-SA 4.0
2018-07-10T16:27:24.533Z
@Blaszard As far as I know, there's no official list with the total number of HSK characters. I just picked the official HSK vocabulary list (since it's official, I'd call it authoritative), removed the repeated characters and came with that number: 2635 individual characters. I also counted how many extra pronunciations there were, which adds another 101 characters to the list.
Enrico Brasil
CC BY-SA 4.0
2018-08-30T15:23:26.950Z
I realised a while ago that there are plenty of useful characters that aren't in the HSK lists. It could be useful for students who have already gone through the HSK lists to have a list of the top, say, 1000 characters in Jun Da's list that aren't in the HSK lists.
goPlayerJuggler
<characters><vocabulary><hsk>
Why do HSK lists differ from character frequency lists?
# Why do HSK lists differ from character frequency lists?

(Asked by: username_0 on 2018-05-10)

I have been wondering how come the HSK list be so different from the list of the most used Chinese characters.

The only list I know about character frequency is [Jun Da's list](http://lingua.mtsu.edu/chinese-computing/statistics/char/list.php?Which=MO). On the first 2635 most common characters of the list (I picked 2635, because it's the total number of the HSK characters), there are exactly 305 characters that are not listed on any HSK.

On the rest of Jun Da's list (a total of 7000 characters) there are another 304 characters that are on the HSK list, but are way way farther than the most used characters (the last two rank at #4527 and #4771).

**Questions:**

1. What are the criteria for the character choice on HSK?
2. Why is the HSK list so different from the most used characters list?

Additional comments are very welcome.

**Tags:** characters, vocabulary, hsk

> **Comments:**
> **username_1**: What do you mean by *"I picked 2635, because it's the total number of the HSK characters"*? I googled by "hsk number of characters 2635" and this post showed at the top of the result, so I'm sure the figure is not authoritative at all.  
> **username_0**: @username_1 As far as I know, there's no official list with the total number of HSK characters. I just picked the official HSK vocabulary list (since it's official, I'd call it authoritative), removed the repeated characters and came with that number: 2635 individual characters. I also counted how many extra pronunciations there were, which adds another 101 characters to the list.  
> **username_2**: I realised a while ago that there are plenty of useful characters that aren't in the HSK lists. It could be useful for students who have already gone through the HSK lists to have a list of the top, say, 1000 characters in Jun Da's list that aren't in the HSK lists.  

---

## 2 answers

### Answer by username_3 on 2018-07-30. Score: 3

To answer your second question:

From a mathematical perspective, it will depend on the texts that are sampled to compile any list and the variances associated with those samples. Languages evolve and there is no true character frequency, only an approximation. Spoken Chinese will have a different character frequency to written Chinese because the spoken language has a high entropy. It is likely that the HSK test had a panel of experts select characters based on what they deemed to be suitable, whereas the list you referenced was mass sampled by some kind of web crawler.

It is inconceivable that every single electronic Chinese character on the internet has been sampled, and if it was that would be very biased as that would ignore all of the written works not uploaded to the internet. Depending on the format of the inputs there will be a huge difference, if lots of text was taken from 微薄, then the character 哈 might rank as common, purely from comments that are like "hahahaha" for example. If classical Chinese was sampled heavily then characters like 曰 might feature much more frequently than 说.

Some of the HSK 5 and 6 words are very political and business oriented, but there are not many scientific words.

> **Comments:**
> **username_0**: It's mostly made of formal written modern Chinese, as you can see here: http://lingua.mtsu.edu/chinese-computing/old-version1/docs/index.html . You have a nice point of view, nonetheless.  

---

### Answer by username_4 on 2020-08-19. Score: 2

To give some partial information regarding your questions, username_5 at Hacking Chinese has a recent (July 2020) article [What important words are missing from HSK?](https://www.hackingchinese.com/what-important-words-are-missing-from-hsk/). One thing he mentions is that:

> It should be clear that HSK is not meant to be a representation of the
> most commonly used Chinese words. This is very obvious in the lower
> levels, where words like “train station” and “bus” are part of HSK1,
> which has only 150 words in total. Those words are nowhere near the
> top 150 words in Chinese in general, but they are of course important
> for foreigners visiting and travelling in China, which probably is why
> they are included.
>
> Overall, I think the lower levels of HSK match the needs of foreign
> students quite well.

Thus, one possibility is that the creators wanted to make sure that foreigners learned what they needed to get around in China when visiting/studying there. This would bias the most frequently used words away from more general frequency lists.

This being said, Olle also came up with a list of words from a frequency list gleaned from movie and TV subtitles (the SUBTLEX-CH corpus [Cai and Brysbaert, 2010]) that either do not appear in HSK or are delayed (ie. they are in HSK but appear in a higher HSK level than expected, based on their frequency). Based on this analysis, Olle found that these types of words seemed to be missing from the HSK lists:

* Many single-character words are missing
* Names of places and countries are missing
* Regional variants are missing
* Profanity is missing entirely
* Foreign things are mostly missing
* Particles in informal language are missing

Seeing what is missing from HSK could help give some insight into the decision making process behind making the lists. Olle goes into more detail about the types of words missing in his article, if you want to know more.

Also, Olle did the same analysis for TOCFL (though with a different frequency list): [What important words are missing or delayed in TOCFL?](https://www.hackingchinese.com/what-important-words-are-missing-from-tocfl/)

**An important practical note:** if you want to learn the words that are "missing" from the HSK (or TOCFL) lists, Olle has generated the word lists and placed them at the end of the article so that people can use them freely for their own study.

Frequency List Reference:

Cai, Q., & Brysbaert, M. (2010). SUBTLEX-CH: Chinese word and character frequencies based on film subtitles. PloS one, 5(6), e10729.

> **Comments:**
> **username_5**: Some extra info: I chose to do the analysis you refer to based on movie subtitles because that's closer to what foreigners want to talk about. I also did (but never published) an analysis comparing HSK to Junda's frequency list. That's not very interesting, because it's basically a list of formal or written Chinese that obviously is very common in Chinese, but not suitable for beginners. This is not terribly interesting, so I focused on the SUBTLEX-CH corpus, which turned out to be quite interesting!  

---

