  Object segmentation requires both object-level information and low-level
pixel data. This presents a challenge for feedforward networks: lower layers in
convolutional nets capture rich spatial information, while upper layers encode
object-level knowledge but are invariant to factors such as pose and
appearance. In this work we propose to augment feedforward nets for object
segmentation with a novel top-down refinement approach. The resulting
bottom-up/top-down architecture is capable of efficiently generating
high-fidelity object masks. Similarly to skip connections, our approach
leverages features at all layers of the net. Unlike skip connections, our
approach does not attempt to output independent predictions at each layer.
Instead, we first output a coarse `mask encoding' in a feedforward pass, then
refine this mask encoding in a top-down pass utilizing features at successively
lower layers. The approach is simple, fast, and effective. Building on the
recent DeepMask network for generating object proposals, we show accuracy
improvements of 10-20% in average recall for various setups. Additionally, by
optimizing the overall network architecture, our approach, which we call
SharpMask, is 50% faster than the original DeepMask network (under .8s per
image).
