  Estimating camera motion in deformable scenes poses a complex and open
research challenge. Most existing non-rigid structure from motion techniques
assume to observe also static scene parts besides deforming scene parts in
order to establish an anchoring reference. However, this assumption does not
hold true in certain relevant application cases such as endoscopies. Deformable
odometry and SLAM pipelines, which tackle the most challenging scenario of
exploratory trajectories, suffer from a lack of robustness and proper
quantitative evaluation methodologies. To tackle this issue with a common
benchmark, we introduce the Drunkard's Dataset, a challenging collection of
synthetic data targeting visual navigation and reconstruction in deformable
environments. This dataset is the first large set of exploratory camera
trajectories with ground truth inside 3D scenes where every surface exhibits
non-rigid deformations over time. Simulations in realistic 3D buildings lets us
obtain a vast amount of data and ground truth labels, including camera poses,
RGB images and depth, optical flow and normal maps at high resolution and
quality. We further present a novel deformable odometry method, dubbed the
Drunkard's Odometry, which decomposes optical flow estimates into rigid-body
camera motion and non-rigid scene deformations. In order to validate our data,
our work contains an evaluation of several baselines as well as a novel
tracking error metric which does not require ground truth data. Dataset and
code: https://davidrecasens.github.io/TheDrunkard'sOdometry/
