  Recurrent Neural Networks (RNNs) have become the state-of-the-art choice for
extracting patterns from temporal sequences. However, current RNN models are
ill-suited to process irregularly sampled data triggered by events generated in
continuous time by sensors or other neurons. Such data can occur, for example,
when the input comes from novel event-driven artificial sensors that generate
sparse, asynchronous streams of events or from multiple conventional sensors
with different update intervals. In this work, we introduce the Phased LSTM
model, which extends the LSTM unit by adding a new time gate. This gate is
controlled by a parametrized oscillation with a frequency range that produces
updates of the memory cell only during a small percentage of the cycle. Even
with the sparse updates imposed by the oscillation, the Phased LSTM network
achieves faster convergence than regular LSTMs on tasks which require learning
of long sequences. The model naturally integrates inputs from sensors of
arbitrary sampling rates, thereby opening new areas of investigation for
processing asynchronous sensory events that carry timing information. It also
greatly improves the performance of LSTMs in standard RNN applications, and
does so with an order-of-magnitude fewer computes at runtime.
