  Few-shot object detection has made substantial progressby representing novel
class objects using the feature representation learned upon a set of base class
objects. However,an implicit contradiction between novel class classification
and representation is unfortunately ignored. On the one hand, to achieve
accurate novel class classification, the distributions of either two base
classes must be far away fromeach other (max-margin). On the other hand, to
precisely represent novel classes, the distributions of base classes should be
close to each other to reduce the intra-class distance of novel classes
(min-margin). In this paper, we propose a class margin equilibrium (CME)
approach, with the aim to optimize both feature space partition and novel class
reconstruction in a systematic way. CME first converts the few-shot detection
problem to the few-shot classification problem by using a fully connected layer
to decouple localization features. CME then reserves adequate margin space for
novel classes by introducing simple-yet-effective class margin loss during
feature learning. Finally, CME pursues margin equilibrium by disturbing the
features of novel class instances in an adversarial min-max fashion.
Experiments on Pascal VOC and MS-COCO datasets show that CME significantly
improves upon two baseline detectors (up to $3\sim 5\%$ in average), achieving
state-of-the-art performance. Code is available at
https://github.com/Bohao-Lee/CME .
