<p><a href="https://pypi.python.org/pypi/zhon/0.2.1" rel="nofollow">zhon</a> would cover this I believe</p>

<p>Example:</p>

<pre><code>        pinyin_a_re.findall('Yǒurén diūshīle yī bǎ fǔzi, zěnme zhǎo yě méiyǒu zhǎodào.')
    ['Yǒu', 'rén', ' ', 'diū', 'shī', 'le', ' ', 'yī', ' ', 'bǎ', ' ', 'fǔ', 'zi', ',', ' ', 'zěn', 'me', ' ', 'zhǎo', ' ', 'yě', ' ', 'méi', 'yǒu', ' ', 'zhǎo', 'dào', '.']   
</code></pre>

CC BY-SA 3.0
2013-08-08T04:45:16.617Z
Not sure if this works for cases such as splitting "changan" to "chang an" and "langan" to "lan gan" instead on "lang an"?
going
CC BY-SA 3.0
2013-08-08T16:39:53.443Z
@xiaohouzi79 Those are legitimately ambiguous cases (hence the apostrophes you see in "proper" spelling of _Cháng'ān_). The default (no apostrophe) case should be "changan" -> "chan" + "gan", even though that's almost certainly wrong semantically.
Stumpy Joe Pete
CC BY-SA 3.0
2013-08-08T03:15:53.170Z
2013-08-08T03:15:53.170Z
50-3
<p>Does anyone know of any open source code for segmenting Pinyin syllables that has diacritic tone marks?  </p>

<p>Example: </p>

<p>yìrúfǎnzhǎng -> yì rú fǎn zhǎng</p>

<p>C#, Python, or Lex is preferred but any language will do.</p>

<p>I've searched the internet but I'm coming up dry.  </p>

<p><em>Edit:</em> This is the algorithm I'm thinking of implementing myself:  </p>

<p>Sort the standard list of all 414 syllables * 5 (tones) distinct syllable+tone combinations by length.  Starting at the beginning of the text, look for the longest pinyin syllable that matches.  Once a match is found move to the end of the syllable and repeat.  Of course, there is the problem with syllables like xian matching xi'an.  This happens because xian is longer than xi.  For short passages of text and given the expected number of syllables I think I can figure this out by backtracking.  </p>

CC BY-SA 3.0
2013-08-07T22:45:45.343Z
I think you are on the wrong site. This site deals with the language and its usage, so anything to do with programming would be completely offtopic. You should try stackoverflow. That being said, you might want to show your own research there, otherwise your question might get closed.
deutschZuid
CC BY-SA 3.0
2013-08-07T23:09:58.403Z
I think it is more likely that someone in the Chinese StackExchange knows the answer to this question than anyone monitoring StackOverflow.  StackOverflow doesn't even have tags for pinyin, mandarin, or chinese.  This question is VERY specific to Mandarin.  I've been studying Chinese for over a decade and I use programming to process Chinese all the time.  It is very likely that there are others out there like me on the Chinese StackExchange.
stuckintheshuck
CC BY-SA 3.0
2013-08-08T01:10:21.723Z
I can't find a downloadable piece of software that does this, but it is reasonably easy for a programmer to do this (although it can't be done with 100% accuracy). If you want to attempt this yourself and then state where you are stuck then I can assist. If you are just going to wait for a solution to show up your question will likely not end up with any answers. So if you are serious about this have a go. I will check back in 24 hours to see if you want to keep going with this question.
going
CC BY-SA 3.0
2013-08-08T02:57:20.907Z
It is quite easy to code it yourself once you know what are the possible combinations of initials and finals in [pinyin](http://en.wikipedia.org/wiki/Pinyin).
杨以轩
CC BY-SA 3.0
2013-08-08T03:37:10.590Z
@stuckintheshuck I never said people here aren't capable of giving an answer to this; I am just saying it's offtopic for this site.
deutschZuid
CC BY-SA 3.0
2013-08-08T11:11:32.147Z
**To everyone**: This question is quite borderline and I was on the side of closing it because your main point is programming-related. Nevertheless, someone made me notice the fact that the question is asking about **Pinyin-related** stuff and this makes it a less clear-cut case. I won't close it, but if other moderators or if the community close it, then I'll agree with the decision as well.
Alenanno
<pinyin>
Segmentation of Pinyin syllables with diacritics
# Segmentation of Pinyin syllables with diacritics

(Asked by: username_0 on 2013-08-07)

Does anyone know of any open source code for segmenting Pinyin syllables that has diacritic tone marks?

Example:

yìrúfǎnzhǎng -> yì rú fǎn zhǎng

C#, Python, or Lex is preferred but any language will do.

I've searched the internet but I'm coming up dry.

*Edit:* This is the algorithm I'm thinking of implementing myself:

Sort the standard list of all 414 syllables \* 5 (tones) distinct syllable+tone combinations by length. Starting at the beginning of the text, look for the longest pinyin syllable that matches. Once a match is found move to the end of the syllable and repeat. Of course, there is the problem with syllables like xian matching xi'an. This happens because xian is longer than xi. For short passages of text and given the expected number of syllables I think I can figure this out by backtracking.

**Tags:** pinyin

> **Comments:**
> **username_1**: I think you are on the wrong site. This site deals with the language and its usage, so anything to do with programming would be completely offtopic. You should try stackoverflow. That being said, you might want to show your own research there, otherwise your question might get closed.  
> **username_0**: I think it is more likely that someone in the Chinese StackExchange knows the answer to this question than anyone monitoring StackOverflow.  StackOverflow doesn't even have tags for pinyin, mandarin, or chinese.  This question is VERY specific to Mandarin.  I've been studying Chinese for over a decade and I use programming to process Chinese all the time.  It is very likely that there are others out there like me on the Chinese StackExchange.  
> **username_2**: I can't find a downloadable piece of software that does this, but it is reasonably easy for a programmer to do this (although it can't be done with 100% accuracy). If you want to attempt this yourself and then state where you are stuck then I can assist. If you are just username_2 to wait for a solution to show up your question will likely not end up with any answers. So if you are serious about this have a go. I will check back in 24 hours to see if you want to keep username_2 with this question.  
> **username_3**: It is quite easy to code it yourself once you know what are the possible combinations of initials and finals in [pinyin](http://en.wikipedia.org/wiki/Pinyin).  
> **username_1**: @username_0 I never said people here aren't capable of giving an answer to this; I am just saying it's offtopic for this site.  
> **username_4**: **To everyone**: This question is quite borderline and I was on the side of closing it because your main point is programming-related. Nevertheless, someone made me notice the fact that the question is asking about **Pinyin-related** stuff and this makes it a less clear-cut case. I won't close it, but if other moderators or if the community close it, then I'll agree with the decision as well.  

---

## 1 answers

### Answer by username_5 on 2013-08-08. Score: 3 | Accepted answer

[zhon](https://pypi.python.org/pypi/zhon/0.2.1) would cover this I believe

Example:

```
        pinyin_a_re.findall('Yǒurén diūshīle yī bǎ fǔzi, zěnme zhǎo yě méiyǒu zhǎodào.')
    ['Yǒu', 'rén', ' ', 'diū', 'shī', 'le', ' ', 'yī', ' ', 'bǎ', ' ', 'fǔ', 'zi', ',', ' ', 'zěn', 'me', ' ', 'zhǎo', ' ', 'yě', ' ', 'méi', 'yǒu', ' ', 'zhǎo', 'dào', '.']   

```

> **Comments:**
> **username_2**: Not sure if this works for cases such as splitting "changan" to "chang an" and "langan" to "lan gan" instead on "lang an"?  
> **username_6**: @xiaohouzi79 Those are legitimately ambiguous cases (hence the apostrophes you see in "proper" spelling of _Cháng'ān_). The default (no apostrophe) case should be "changan" -> "chan" + "gan", even though that's almost certainly wrong semantically.  

---

