  Deep neural networks with more parameters and FLOPs have higher capacity and
generalize better to diverse domains. But to be deployed on edge devices, the
model's complexity has to be constrained due to limited compute resource. In
this work, we propose a method to improve the model capacity without increasing
inference-time complexity. Our method is based on an assumption of data
locality: for an edge device, within a short period of time, the input data to
the device are sampled from a single domain with relatively low diversity.
Therefore, it is possible to utilize a specialized, low-complexity model to
achieve good performance in that input domain. To leverage this, we propose
Domain-aware Dynamic Network (DDN), which is a high-capacity dynamic network in
which each layer contains multiple weights. During inference, based on the
input domain, DDN dynamically combines those weights into one single weight
that specializes in the given domain. This way, DDN can keep the inference-time
complexity low but still maintain a high capacity. Experiments show that
without increasing the parameters, FLOPs, and actual latency, DDN achieves up
to 2.6\% higher AP50 than a static network on the BDD100K object-detection
benchmark.
