  Object detection has seen remarkable progress in recent years with the
introduction of Convolutional Neural Networks (CNN). Object detection is a
multi-task learning problem where both the position of the objects in the
images as well as their classes needs to be correctly identified. The idea here
is to maximize the overlap between the ground-truth bounding boxes and the
predictions i.e. the Intersection over Union (IoU). In the scope of work seen
currently in this domain, IoU is approximated by using the Huber loss as a
proxy but this indirect method does not leverage the IoU information and treats
the bounding box as four independent, unrelated terms of regression. This is
not true for a bounding box where the four coordinates are highly correlated
and hold a semantic meaning when taken together. The direct optimization of the
IoU is not possible due to its non-convex and non-differentiable nature. In
this paper, we have formulated a novel loss namely, the Smooth IoU, which
directly optimizes the IoUs for the bounding boxes. This loss has been
evaluated on the Oxford IIIT Pets, Udacity self-driving car, PASCAL VOC, and
VWFS Car Damage datasets and has shown performance gains over the standard
Huber loss.
