  Temporal action detection (TAD) involves the localization and classification
of action instances within untrimmed videos. While standard TAD follows fully
supervised learning with closed-set setting on large training data, recent
zero-shot TAD methods showcase the promising open-set setting by leveraging
large-scale contrastive visual-language (ViL) pretrained models. However,
existing zero-shot TAD methods have limitations on how to properly construct
the strong relationship between two interdependent tasks of localization and
classification and adapt ViL model to video understanding. In this work, we
present ZEETAD, featuring two modules: dual-localization and zero-shot proposal
classification. The former is a Transformer-based module that detects action
events while selectively collecting crucial semantic embeddings for later
recognition. The latter one, CLIP-based module, generates semantic embeddings
from text and frame inputs for each temporal unit. Additionally, we enhance
discriminative capability on unseen classes by minimally updating the frozen
CLIP encoder with lightweight adapters. Extensive experiments on THUMOS14 and
ActivityNet-1.3 datasets demonstrate our approach's superior performance in
zero-shot TAD and effective knowledge transfer from ViL models to unseen action
categories.
