  Lip region-of-interest (ROI) is conventionally used for visual input in the
lipreading task. Few works have adopted the entire face as visual input because
lip-excluded parts of the face are usually considered to be redundant and
irrelevant to visual speech recognition. However, faces contain much more
detailed information than lips, such as speakers' head pose, emotion, identity
etc. We argue that such information might benefit visual speech recognition if
a powerful feature extractor employing the entire face is trained. In this
work, we propose to adopt the entire face for lipreading with self-supervised
learning. AV-HuBERT, an audio-visual multi-modal self-supervised learning
framework, was adopted in our experiments. Our experimental results showed that
adopting the entire face achieved 16% relative word error rate (WER) reduction
on the lipreading task, compared with the baseline method using lip as visual
input. Without self-supervised pretraining, the model with face input achieved
a higher WER than that using lip input in the case of limited training data (30
hours), while a slightly lower WER when using large amount of training data
(433 hours).
