  Value iteration networks (VINs) enable end-to-end learning for planning tasks
by employing a differentiable "planning module" that approximates the value
iteration algorithm. However, long-term planning remains a challenge because
training very deep VINs is difficult. To address this problem, we embed highway
value iteration -- a recent algorithm designed to facilitate long-term credit
assignment -- into the structure of VINs. This improvement augments the
"planning module" of the VIN with three additional components: 1) an "aggregate
gate," which constructs skip connections to improve information flow across
many layers; 2) an "exploration module," crafted to increase the diversity of
information and gradient flow in spatial dimensions; 3) a "filter gate"
designed to ensure safe exploration. The resulting novel highway VIN can be
trained effectively with hundreds of layers using standard backpropagation. In
long-term planning tasks requiring hundreds of planning steps, deep highway
VINs outperform both traditional VINs and several advanced, very deep NNs.
