  This paper proposes a method for recognizing online handwritten mathematical
expressions (OnHME) by building a symbol relation tree (SRT) directly from a
sequence of strokes. A bidirectional recurrent neural network learns from
multiple derived paths of SRT to predict both symbols and spatial relations
between symbols using global context. The recognition system has two parts: a
temporal classifier and a tree connector. The temporal classifier produces an
SRT by recognizing an OnHME pattern. The tree connector splits the SRT into
several sub-SRTs. The final SRT is formed by looking up the best combination
among those sub-SRTs. Besides, we adopt a tree sorting method to deal with
various stroke orders. Recognition experiments indicate that the proposed OnHME
recognition system is competitive to other methods. The recognition system
achieves 44.12% and 41.76% expression recognition rates on the Competition on
Recognition of Online Handwritten Mathematical Expressions (CROHME) 2014 and
2016 testing sets.
