<p>The Mediawiki converter uses a combination of automatic information from the Unicode standard, SCIM tables, and other sources plus manual tweaks to build a set of translation tables.  When going from Traditional to Simplified, some characters have been condensed into one.  Translating back from Simplified to Traditional requires context that a computer is not likely to understand unless you implement a full translation engine, not just a text replacement engine. This is where the manual enhancements come into play.  </p>

<p>Mediawiki has a build system which automatically combines the automatic and manual information into a PHP source file of translations between many different types of Chinese.  It appears to cover dialectical differences for mainland Chinese, Hong Kong, Taiwan, and Singapore. The information is then changed out with PHP's strtr() function which replaces all occurrences of items in a given array with matches in the supplied string, longest matches first.  This longest matching is the key when switching out characters because you have more context from the source to make a better replacement. </p>

<p>Source:
Code which uses mediawiki's conversion functions:
<a href="https://github.com/tszming/mediawiki-zhconverter/blob/master/mediawiki-zhconverter.inc.php" rel="nofollow">https://github.com/tszming/mediawiki-zhconverter/blob/master/mediawiki-zhconverter.inc.php</a></p>

<p>Build file to combine automatic conversion tables with manual tweaks:
<a href="https://phabricator.wikimedia.org/diffusion/MW/browse/master/maintenance/language/zhtable/Makefile.py" rel="nofollow">https://phabricator.wikimedia.org/diffusion/MW/browse/master/maintenance/language/zhtable/Makefile.py</a></p>

<p>Compiled translation tables (large file, 17k lines):
<a href="https://phabricator.wikimedia.org/diffusion/MW/browse/master/includes/ZhConversion.php" rel="nofollow">https://phabricator.wikimedia.org/diffusion/MW/browse/master/includes/ZhConversion.php</a></p>

<p>Conversion tool to load tables for different conversions (used by first source link)
<a href="https://phabricator.wikimedia.org/diffusion/MW/browse/master/languages/classes/LanguageZh.php" rel="nofollow">https://phabricator.wikimedia.org/diffusion/MW/browse/master/languages/classes/LanguageZh.php</a></p>

CC BY-SA 3.0
2015-04-05T08:59:36.957Z
I've updated the links from the old SVN repository to the current one. A lot of work is ongoing for the Chinese language converter, see the recent huge patch: https://phabricator.wikimedia.org/rMWf968119a28dcd3559e0f5f9980040b51deebf459
Nemo
CC BY-SA 3.0
2012-01-25T14:32:14.493Z
2015-04-05T19:21:38.267Z
2015-04-05T19:21:38.267Z
chugadie
<p>There is another converter at <a href="http://mandarintools.com/zhcode.html" rel="nofollow">http://mandarintools.com/zhcode.html</a>. I lack the capacity to judge its quality. Is that a suitable alternative? It may reveal how the conversion is done.</p>

<p>I may have overlooked some parts, but here is my understanding of it:</p>

<ul>
<li>It seems to have a large two-column list of words (<code>hcutf8.txt</code>). The first column contains simplified, the second column traditional. The second column occasionally contains two or more characters, in situations where one simplified character equals more than one traditional character.</li>
<li>When converting from simplified to traditional, it replaces all occurrences of words found in the first column with words in the second column.</li>
<li>When converting from traditional to simplified, it replaces all occurrences of words found in the second column with words in the first column.</li>
</ul>

<p>Perhaps there is more in the code that I did not notice. Are additional steps needed for this conversion to be successful?</p>

CC BY-SA 3.0
2011-12-26T06:15:17.980Z
2011-12-26T06:15:17.980Z
Village
<p>The relationship between simplified and traditional characters can be a headache for <a href="http://meta.wikimedia.org/wiki/Problems_with_server-side_Chinese_to_Chinese_translation_in_MediaWiki" rel="noreferrer">organizations using both orthographies</a>.</p>

<blockquote>
  <p>"A number of surveys, such as [Xiandai 1986], have demonstrated that
  the 2000 most frequent SC characters account for approximately 97% of
  all characters occurring in contemporary SC corpora. Of these, 238
  simplified forms, or almost 12%, are polygraphic; that is, they map to
  two or more traditional forms."</p>
</blockquote>

<p>Nevertheless, it's a task that can one day be done relatively perfectly by a computer. Google Translate can do it for you more or less. But if you are converting a short story or a novel, the task is a little more difficult. That's why <a href="https://unix.stackexchange.com/questions/23830/making-a-simple-php-script-process-text-in-a-pipe">I actually prefer</a> to use <a href="https://github.com/tszming/mediawiki-zhconverter" rel="noreferrer">the Mediawiki converter</a>.</p>

<p>Nevertheless, I don't really know how it works. I have a feeling the SCIM pinyin input tables are used somehow, and the tediously underdocumented non-bijective-mappings between the two orthographies coded for manually. Is that right?</p>

CC BY-SA 3.0
2011-12-25T21:18:04.240Z
Hello Magnetar. Thanks for posting but I don't think your question is on topic, I voted to close. It's more related to how some web-software works rather than the Chinese language. I think it's more fit in sites like Superuser or similar, I'm not sure which one, honestly... :D
Alenanno
CC BY-SA 3.0
2011-12-26T06:16:48.163Z
Do you have a link to all of the source code? Is this project considered completely finished?
Village
CC BY-SA 3.0
2011-12-26T07:17:49.387Z
Would you support this question, @Alenanno, if it didn't focus exclusively on the Mediawiki converter? Something like, "What techniques can software use to convert simplified characters to traditional characters when simplified characters can map to more than one? For example, Mediawiki seems to work well."
Don Kirkby
CC BY-SA 3.0
2011-12-26T11:17:56.553Z
@DonKirkby Uhm It still seems too related to software. Anyway, I wanted to be clear that this questions is not a bad one (and as you can see I didn't down vote), it's an interesting question to me as well, but I think that the knowledge required to answer is more on the computer side than the chinese side.
Alenanno
CC BY-SA 3.0
2012-01-12T01:23:44.227Z
@magnetar - Can you confirm if you want a technical or non-technical answer to your question? Are you looking for someone to explain how this can be done in code or just a general explanation to how it is done?
going
CC BY-SA 3.0
2012-01-18T22:21:12.693Z
@magnetar - Please refer to my comment above and improve your question by editing. The question currently reads like a technical request and the question may be closed as off topic.
going
CC BY-SA 3.0
2012-01-25T14:38:50.007Z
This question is more about the digital representation of various Chinese character systems.  I think to original poster was just looking for confirmation that the systems used to represent Chinese digitally have no easy conversion back and forth.  Just because it is not desktop software does not make it any more technical than questions about using Pinyin input methods.
chugadie
<software><simplified-characters><traditional-characters><traditional-vs-simplified>
How does Wikipedia's simplified to traditional converter work?
# How does Wikipedia's simplified to traditional converter work?

(Asked by: username_0 on 2011-12-25)

The relationship between simplified and traditional characters can be a headache for [organizations using both orthographies](http://meta.wikimedia.org/wiki/Problems_with_server-side_Chinese_to_Chinese_translation_in_MediaWiki).

> "A number of surveys, such as [Xiandai 1986], have demonstrated that
> the 2000 most frequent SC characters account for approximately 97% of
> all characters occurring in contemporary SC corpora. Of these, 238
> simplified forms, or almost 12%, are polygraphic; that is, they map to
> two or more traditional forms."

Nevertheless, it's a task that can one day be done relatively perfectly by a computer. Google Translate can do it for you more or less. But if you are converting a short story or a novel, the task is a little more difficult. That's why [I actually prefer](https://unix.stackexchange.com/questions/23830/making-a-simple-php-script-process-text-in-a-pipe) to use [the Mediawiki converter](https://github.com/tszming/mediawiki-zhconverter).

Nevertheless, I don't really know how it works. I have a feeling the SCIM pinyin input tables are used somehow, and the tediously underdocumented non-bijective-mappings between the two orthographies coded for manually. Is that right?

**Tags:** software, simplified-characters, traditional-characters, traditional-vs-simplified

> **Comments:**
> **username_1**: Hello username_0. Thanks for posting but I don't think your question is on topic, I voted to close. It's more related to how some web-software works rather than the Chinese language. I think it's more fit in sites like Superuser or similar, I'm not sure which one, honestly... :D  
> **username_2**: Do you have a link to all of the source code? Is this project considered completely finished?  
> **username_3**: Would you support this question, @username_1, if it didn't focus exclusively on the Mediawiki converter? Something like, "What techniques can software use to convert simplified characters to traditional characters when simplified characters can map to more than one? For example, Mediawiki seems to work well."  
> **username_1**: @username_3 Uhm It still seems too related to software. Anyway, I wanted to be clear that this questions is not a bad one (and as you can see I didn't down vote), it's an interesting question to me as well, but I think that the knowledge required to answer is more on the computer side than the chinese side.  
> **username_4**: @username_0 - Can you confirm if you want a technical or non-technical answer to your question? Are you looking for someone to explain how this can be done in code or just a general explanation to how it is done?  
> **username_4**: @username_0 - Please refer to my comment above and improve your question by editing. The question currently reads like a technical request and the question may be closed as off topic.  
> **username_5**: This question is more about the digital representation of various Chinese character systems.  I think to original poster was just looking for confirmation that the systems used to represent Chinese digitally have no easy conversion back and forth.  Just because it is not desktop software does not make it any more technical than questions about using Pinyin input methods.  

---

## 2 answers

### Answer by username_5 on 2012-01-25. Score: 7

The Mediawiki converter uses a combination of automatic information from the Unicode standard, SCIM tables, and other sources plus manual tweaks to build a set of translation tables. When username_4 from Traditional to Simplified, some characters have been condensed into one. Translating back from Simplified to Traditional requires context that a computer is not likely to understand unless you implement a full translation engine, not just a text replacement engine. This is where the manual enhancements come into play.

Mediawiki has a build system which automatically combines the automatic and manual information into a PHP source file of translations between many different types of Chinese. It appears to cover dialectical differences for mainland Chinese, Hong Kong, Taiwan, and Singapore. The information is then changed out with PHP's strtr() function which replaces all occurrences of items in a given array with matches in the supplied string, longest matches first. This longest matching is the key when switching out characters because you have more context from the source to make a better replacement.

Source:
Code which uses mediawiki's conversion functions:
<https://github.com/tszming/mediawiki-zhconverter/blob/master/mediawiki-zhconverter.inc.php>

Build file to combine automatic conversion tables with manual tweaks:
<https://phabricator.wikimedia.org/diffusion/MW/browse/master/maintenance/language/zhtable/Makefile.py>

Compiled translation tables (large file, 17k lines):
<https://phabricator.wikimedia.org/diffusion/MW/browse/master/includes/ZhConversion.php>

Conversion tool to load tables for different conversions (used by first source link)
<https://phabricator.wikimedia.org/diffusion/MW/browse/master/languages/classes/LanguageZh.php>

> **Comments:**
> **username_6**: I've updated the links from the old SVN repository to the current one. A lot of work is ongoing for the Chinese language converter, see the recent huge patch: https://phabricator.wikimedia.org/rMWf968119a28dcd3559e0f5f9980040b51deebf459  

---

### Answer by username_2 on 2011-12-26. Score: 1

There is another converter at <http://mandarintools.com/zhcode.html>. I lack the capacity to judge its quality. Is that a suitable alternative? It may reveal how the conversion is done.

I may have overlooked some parts, but here is my understanding of it:

* It seems to have a large two-column list of words (`hcutf8.txt`). The first column contains simplified, the second column traditional. The second column occasionally contains two or more characters, in situations where one simplified character equals more than one traditional character.
* When converting from simplified to traditional, it replaces all occurrences of words found in the first column with words in the second column.
* When converting from traditional to simplified, it replaces all occurrences of words found in the second column with words in the first column.

Perhaps there is more in the code that I did not notice. Are additional steps needed for this conversion to be successful?


---

