  In this work we introduce a convolutional neural network (CNN) that jointly
handles low-, mid-, and high-level vision tasks in a unified architecture that
is trained end-to-end. Such a universal network can act like a `swiss knife'
for vision tasks; we call this architecture an UberNet to indicate its
overarching nature.
  We address two main technical challenges that emerge when broadening up the
range of tasks handled by a single CNN: (i) training a deep architecture while
relying on diverse training sets and (ii) training many (potentially unlimited)
tasks with a limited memory budget. Properly addressing these two problems
allows us to train accurate predictors for a host of tasks, without
compromising accuracy.
  Through these advances we train in an end-to-end manner a CNN that
simultaneously addresses (a) boundary detection (b) normal estimation (c)
saliency estimation (d) semantic segmentation (e) human part segmentation (f)
semantic boundary detection, (g) region proposal generation and object
detection. We obtain competitive performance while jointly addressing all of
these tasks in 0.7 seconds per frame on a single GPU. A demonstration of this
system can be found at http://cvn.ecp.fr/ubernet/.
