  The dominant object detection approaches treat the recognition of each region
separately and overlook crucial semantic correlations between objects in one
scene. This paradigm leads to substantial performance drop when facing heavy
long-tail problems, where very few samples are available for rare classes and
plenty of confusing categories exists. We exploit diverse human commonsense
knowledge for reasoning over large-scale object categories and reaching
semantic coherency within one image. Particularly, we present Hybrid Knowledge
Routed Modules (HKRM) that incorporates the reasoning routed by two kinds of
knowledge forms: an explicit knowledge module for structured constraints that
are summarized with linguistic knowledge (e.g. shared attributes,
relationships) about concepts; and an implicit knowledge module that depicts
some implicit constraints (e.g. common spatial layouts). By functioning over a
region-to-region graph, both modules can be individualized and adapted to
coordinate with visual patterns in each image, guided by specific knowledge
forms. HKRM are light-weight, general-purpose and extensible by easily
incorporating multiple knowledge to endow any detection networks the ability of
global semantic reasoning. Experiments on large-scale object detection
benchmarks show HKRM obtains around 34.5% improvement on VisualGenome (1000
categories) and 30.4% on ADE in terms of mAP. Codes and trained model can be
found in https://github.com/chanyn/HKRM.
