  We explore object detection with two attributes: color and material. The task
aims to simultaneously detect objects and infer their color and material. A
straight-forward approach is to add attribute heads at the very end of a usual
object detection pipeline. However, we observe that the two goals are in
conflict: Object detection should be attribute-independent and attributes be
largely object-independent. Features computed by a standard detection network
entangle the category and attribute features; we disentangle them by the use of
a two-stream model where the category and attribute features are computed
independently but the classification heads share Regions of Interest (RoIs).
Compared with a traditional single-stream model, our model shows significant
improvements over VG-20, a subset of Visual Genome, on both supervised and
attribute transfer tasks.
