  Text recognition in natural scene is a challenging problem due to the many
factors affecting text appearance. In this paper, we presents a method that
directly transcribes scene text images to text without needing of sophisticated
character segmentation. We leverage recent advances of deep neural networks to
model the appearance of scene text images with temporal dynamics. Specifically,
we integrates convolutional neural network (CNN) and recurrent neural network
(RNN) which is motivated by observing the complementary modeling capabilities
of the two models. The main contribution of this work is investigating how
temporal memory helps in an segmentation free fashion for this specific
problem. By using long short-term memory (LSTM) blocks as hidden units, our
model can retain long-term memory compared with HMMs which only maintain
short-term state dependences. We conduct experiments on Street View House
Number dataset containing highly variable number images. The results
demonstrate the superiority of the proposed method over traditional HMM based
methods.
