  Training on datasets with long-tailed distributions has been challenging for
major recognition tasks such as classification and detection. To deal with this
challenge, image resampling is typically introduced as a simple but effective
approach. However, we observe that long-tailed detection differs from
classification since multiple classes may be present in one image. As a result,
image resampling alone is not enough to yield a sufficiently balanced
distribution at the object level. We address object-level resampling by
introducing an object-centric memory replay strategy based on dynamic, episodic
memory banks. Our proposed strategy has two benefits: 1) convenient
object-level resampling without significant extra computation, and 2) implicit
feature-level augmentation from model updates. We show that image-level and
object-level resamplings are both important, and thus unify them with a joint
resampling strategy (RIO). Our method outperforms state-of-the-art long-tailed
detection and segmentation methods on LVIS v0.5 across various backbones. Code
is available at https://github.com/NVlabs/RIO.
