  Fully supervised deep neural networks for segmentation usually require a
massive amount of pixel-level labels which are manually expensive to create. In
this work, we develop a multi-task learning method to relax this constraint. We
regard the segmentation problem as a sequence of approximation subproblems that
are recursively defined and in increasing levels of approximation accuracy. The
subproblems are handled by a framework that consists of 1) a segmentation task
that learns from pixel-level ground truth segmentation masks of a small
fraction of the images, 2) a recursive approximation task that conducts partial
object regions learning and data-driven mask evolution starting from partial
masks of each object instance, and 3) other problem oriented auxiliary tasks
that are trained with sparse annotations and promote the learning of dedicated
features. Most training images are only labeled by (rough) partial masks, which
do not contain exact object boundaries, rather than by their full segmentation
masks. During the training phase, the approximation task learns the statistics
of these partial masks, and the partial regions are recursively increased
towards object boundaries aided by the learned information from the
segmentation task in a fully data-driven fashion. The network is trained on an
extremely small amount of precisely segmented images and a large set of coarse
labels. Annotations can thus be obtained in a cheap way. We demonstrate the
efficiency of our approach in three applications with microscopy images and
ultrasound images.
