  Event cameras respond to scene dynamics and offer advantages to estimate
motion. Following recent image-based deep-learning achievements, optical flow
estimation methods for event cameras have rushed to combine those image-based
methods with event data. However, it requires several adaptations (data
conversion, loss function, etc.) as they have very different properties. We
develop a principled method to extend the Contrast Maximization framework to
estimate optical flow from events alone. We investigate key elements: how to
design the objective function to prevent overfitting, how to warp events to
deal better with occlusions, and how to improve convergence with multi-scale
raw events. With these key elements, our method ranks first among unsupervised
methods on the MVSEC benchmark, and is competitive on the DSEC benchmark.
Moreover, our method allows us to expose the issues of the ground truth flow in
those benchmarks, and produces remarkable results when it is transferred to
unsupervised learning settings. Our code is available at
https://github.com/tub-rip/event_based_optical_flow
