  We focus on the word-level visual lipreading, which requires to decode the
word from the speaker's video. Recently, many state-of-the-art visual
lipreading methods explore the end-to-end trainable deep models, involving the
use of 2D convolutional networks (e.g., ResNet) as the front-end visual feature
extractor and the sequential model (e.g., Bi-LSTM or Bi-GRU) as the back-end.
Although a deep 2D convolution neural network can provide informative
image-based features, it ignores the temporal motion existing between the
adjacent frames. In this work, we investigate the spatial-temporal capacity
power of I3D (Inflated 3D ConvNet) for visual lipreading. We demonstrate that,
after pre-trained on the large-scale video action recognition dataset (e.g.,
Kinetics), our models show a considerable improvement of performance on the
task of lipreading. A comparison between a set of video model architectures and
input data representation is also reported. Our extensive experiments on LRW
shows that a two-stream I3D model with RGB video and optical flow as the inputs
achieves the state-of-the-art performance.
