  Recurrent neural network (RNN), as a powerful contextual dependency modeling
framework, has been widely applied to scene labeling problems. However, this
work shows that directly applying traditional RNN architectures, which unfolds
a 2D lattice grid into a sequence, is not sufficient to model structure
dependencies in images due to the "impact vanishing" problem. First, we give an
empirical analysis about the "impact vanishing" problem. Then, a new RNN unit
named Recurrent Neural Network with explicit long range conditioning (RNN-ELC)
is designed to alleviate this problem. A novel neural network architecture is
built for scene labeling tasks where one of the variants of the new RNN unit,
Gated Recurrent Unit with Explicit Long-range Conditioning (GRU-ELC), is used
to model multi scale contextual dependencies in images. We validate the use of
GRU-ELC units with state-of-the-art performance on three standard scene
labeling datasets. Comprehensive experiments demonstrate that the new GRU-ELC
unit benefits scene labeling problem a lot as it can encode longer contextual
dependencies in images more effectively than traditional RNN units.
