  Open vocabulary object detection has been greatly advanced by the recent
development of vision-language pretrained model, which helps recognize novel
objects with only semantic categories. The prior works mainly focus on
knowledge transferring to the object proposal classification and employ
class-agnostic box and mask prediction. In this work, we propose CondHead, a
principled dynamic network design to better generalize the box regression and
mask segmentation for open vocabulary setting. The core idea is to
conditionally parameterize the network heads on semantic embedding and thus the
model is guided with class-specific knowledge to better detect novel
categories. Specifically, CondHead is composed of two streams of network heads,
the dynamically aggregated head and the dynamically generated head. The former
is instantiated with a set of static heads that are conditionally aggregated,
these heads are optimized as experts and are expected to learn sophisticated
prediction. The latter is instantiated with dynamically generated parameters
and encodes general class-specific information. With such a conditional design,
the detection model is bridged by the semantic embedding to offer strongly
generalizable class-wise box and mask prediction. Our method brings significant
improvement to the state-of-the-art open vocabulary object detection methods
with very minor overhead, e.g., it surpasses a RegionClip model by 3.0
detection AP on novel categories, with only 1.1% more computation.
