  In this paper we revisit feature fusion, an old-fashioned topic, in the new
context of text-to-video retrieval. Different from previous research that
considers feature fusion only at one end, let it be video or text, we aim for
feature fusion for both ends within a unified framework. We hypothesize that
optimizing the convex combination of the features is preferred to modeling
their correlations by computationally heavy multi-head self attention. We
propose Lightweight Attentional Feature Fusion (LAFF). LAFF performs feature
fusion at both early and late stages and at both video and text ends, making it
a powerful method for exploiting diverse (off-the-shelf) features. The
interpretability of LAFF can be used for feature selection. Extensive
experiments on five public benchmark sets (MSR-VTT, MSVD, TGIF, VATEX and
TRECVID AVS 2016-2020) justify LAFF as a new baseline for text-to-video
retrieval.
