  Current deep video quality assessment (VQA) methods are usually with high
computational costs when evaluating high-resolution videos. This cost hinders
them from learning better video-quality-related representations via end-to-end
training. Existing approaches typically consider naive sampling to reduce the
computational cost, such as resizing and cropping. However, they obviously
corrupt quality-related information in videos and are thus not optimal for
learning good representations for VQA. Therefore, there is an eager need to
design a new quality-retained sampling scheme for VQA. In this paper, we
propose Grid Mini-patch Sampling (GMS), which allows consideration of local
quality by sampling patches at their raw resolution and covers global quality
with contextual relations via mini-patches sampled in uniform grids. These
mini-patches are spliced and aligned temporally, named as fragments. We further
build the Fragment Attention Network (FANet) specially designed to accommodate
fragments as inputs. Consisting of fragments and FANet, the proposed FrAgment
Sample Transformer for VQA (FAST-VQA) enables efficient end-to-end deep VQA and
learns effective video-quality-related representations. It improves
state-of-the-art accuracy by around 10% while reducing 99.5% FLOPs on 1080P
high-resolution videos. The newly learned video-quality-related representations
can also be transferred into smaller VQA datasets, boosting performance in
these scenarios. Extensive experiments show that FAST-VQA has good performance
on inputs of various resolutions while retaining high efficiency. We publish
our code at https://github.com/timothyhtimothy/FAST-VQA.
