  We address the problem of joint optical flow and camera motion estimation in
rigid scenes by incorporating geometric constraints into an unsupervised deep
learning framework. Unlike existing approaches which rely on brightness
constancy and local smoothness for optical flow estimation, we exploit the
global relationship between optical flow and camera motion using epipolar
geometry. In particular, we formulate the prediction of optical flow and camera
motion as a bi-level optimization problem, consisting of an upper-level problem
to estimate the flow that conforms to the predicted camera motion, and a
lower-level problem to estimate the camera motion given the predicted optical
flow. We use implicit differentiation to enable back-propagation through the
lower-level geometric optimization layer independent of its implementation,
allowing end-to-end training of the network. With globally-enforced geometric
constraints, we are able to improve the quality of the estimated optical flow
in challenging scenarios and obtain better camera motion estimates compared to
other unsupervised learning methods.
