  Recent differentiable rendering techniques have become key tools to tackle
many inverse problems in graphics and vision. Existing models, however, assume
steady-state light transport, i.e., infinite speed of light. While this is a
safe assumption for many applications, recent advances in ultrafast imaging
leverage the wealth of information that can be extracted from the exact time of
flight of light. In this context, physically-based transient rendering allows
to efficiently simulate and analyze light transport considering that the speed
of light is indeed finite. In this paper, we introduce a novel differentiable
transient rendering framework, to help bring the potential of differentiable
approaches into the transient regime. To differentiate the transient path
integral we need to take into account that scattering events at path vertices
are no longer independent; instead, tracking the time of flight of light
requires treating such scattering events at path vertices jointly as a
multidimensional, evolving manifold. We thus turn to the generalized transport
theorem, and introduce a novel correlated importance term, which links the
time-integrated contribution of a path to its light throughput, and allows us
to handle discontinuities in the light and sensor functions. Last, we present
results in several challenging scenarios where the time of flight of light
plays an important role such as optimizing indices of refraction,
non-line-of-sight tracking with nonplanar relay walls, and non-line-of-sight
tracking around two corners.
