  The need to recognise long-term dependencies in sequential data such as video
streams has made Long Short-Term Memory (LSTM) networks a prominent Artificial
Intelligence model for many emerging applications. However, the high
computational and memory demands of LSTMs introduce challenges in their
deployment on latency-critical systems such as self-driving cars which are
equipped with limited computational resources on-board. In this paper, we
introduce a progressive inference computing scheme that combines model pruning
and computation restructuring leading to the best possible approximation of the
result given the available latency budget of the target application. The
proposed methodology enables mission-critical systems to make informed
decisions even in early stages of the computation, based on approximate LSTM
inference, meeting their specifications on safety and robustness. Our
experiments on a state-of-the-art driving model for autonomous vehicle
navigation demonstrate that the proposed approach can yield outputs with
similar quality of result compared to a faithful LSTM baseline, up to 415x
faster (198x on average, 76x geo. mean).
