  This paper presents a novel architecture for simultaneous estimation of
highly accurate optical flows and rigid scene transformations for difficult
scenarios where the brightness assumption is violated by strong shading
changes. In the case of rotating objects or moving light sources, such as those
encountered for driving cars in the dark, the scene appearance often changes
significantly from one view to the next. Unfortunately, standard methods for
calculating optical flows or poses are based on the expectation that the
appearance of features in the scene remain constant between views. These
methods may fail frequently in the investigated cases. The presented method
fuses texture and geometry information by combining image, vertex and normal
data to compute an illumination-invariant optical flow. By using a
coarse-to-fine strategy, globally anchored optical flows are learned, reducing
the impact of erroneous shading-based pseudo-correspondences. Based on the
learned optical flows, a second architecture is proposed that predicts robust
rigid transformations from the warped vertex and normal maps. Particular
attention is payed to situations with strong rotations, which often cause such
shading changes. Therefore a 3-step procedure is proposed that profitably
exploits correlations between the normals and vertices. The method has been
evaluated on a newly created dataset containing both synthetic and real data
with strong rotations and shading effects. This data represents the typical use
case in 3D reconstruction, where the object often rotates in large steps
between the partial reconstructions. Additionally, we apply the method to the
well-known Kitti Odometry dataset. Even if, due to fulfillment of the brighness
assumption, this is not the typical use case of the method, the applicability
to standard situations and the relation to other methods is therefore
established.
