  Event cameras open up new possibilities for robotic perception due to their
low latency and high dynamic range. On the other hand, developing effective
event-based vision algorithms that fully exploit the beneficial properties of
event cameras remains work in progress. In this paper, we focus on event-based
visual odometry (VO). While existing event-driven VO pipelines have adopted
continuous-time representations to asynchronously process event data, they
either assume a known map, restrict the camera to planar trajectories, or
integrate other sensors into the system. Towards map-free event-only monocular
VO in SE(3), we propose an asynchronous structure-from-motion optimisation
back-end. Our formulation is underpinned by a principled joint optimisation
problem involving non-parametric Gaussian Process motion modelling and
incremental maximum a posteriori inference. A high-performance incremental
computation engine is employed to reason about the camera trajectory with every
incoming event. We demonstrate the robustness of our asynchronous back-end in
comparison to frame-based methods which depend on accurate temporal
accumulation of measurements.
