  In this project, we worked on speech recognition, specifically predicting
individual words based on both the video frames and audio. Empowered by
convolutional neural networks, the recent speech recognition and lip reading
models are comparable to human level performance. We re-implemented and made
derivations of the state-of-the-art model. Then, we conducted rich experiments
including the effectiveness of attention mechanism, more accurate residual
network as the backbone with pre-trained weights and the sensitivity of our
model with respect to audio input with/without noise.
