  Using deep learning, this paper addresses the problem of joint object
boundary detection and boundary motion estimation in videos, which we named
boundary flow estimation. Boundary flow is an important mid-level visual cue as
boundaries characterize objects spatial extents, and the flow indicates objects
motions and interactions. Yet, most prior work on motion estimation has focused
on dense object motion or feature points that may not necessarily reside on
boundaries. For boundary flow estimation, we specify a new fully convolutional
Siamese network (FCSN) that jointly estimates object-level boundaries in two
consecutive frames. Boundary correspondences in the two frames are predicted by
the same FCSN with a new, unconventional deconvolution approach. Finally, the
boundary flow estimate is improved with an edgelet-based filtering. Evaluation
is conducted on three tasks: boundary detection in videos, boundary flow
estimation, and optical flow estimation. On boundary detection, we achieve the
state-of-the-art performance on the benchmark VSB100 dataset. On boundary flow
estimation, we present the first results on the Sintel training dataset. For
optical flow estimation, we run the recent approach CPMFlow but on the
augmented input with our boundary-flow matches, and achieve significant
performance improvement on the Sintel benchmark.
