  Open-vocabulary object detection (OVOD) aims to recognize novel objects whose
categories are not included in the training set. In order to classify these
unseen classes during training, many OVOD frameworks leverage the zero-shot
capability of largely pretrained vision and language models, such as CLIP. To
further improve generalization on the unseen novel classes, several approaches
proposed to additionally train with pseudo region labeling on the external data
sources that contain a substantial number of novel category labels beyond the
existing training data. Albeit its simplicity, these pseudo-labeling methods
still exhibit limited improvement with regard to the truly unseen novel classes
that were not pseudo-labeled. In this paper, we present a novel, yet simple
technique that helps generalization on the overall distribution of novel
classes. Inspired by our observation that numerous novel classes reside within
the convex hull constructed by the base (seen) classes in the CLIP embedding
space, we propose to synthesize proxy-novel classes approximating novel classes
via linear mixup between a pair of base classes. By training our detector with
these synthetic proxy-novel classes, we effectively explore the embedding space
of novel classes. The experimental results on various OVOD benchmarks such as
LVIS and COCO demonstrate superior performance on novel classes compared to the
other state-of-the-art methods. Code is available at
https://github.com/clovaai/ProxyDet.
