  Optimal control theory deals with finding protocols to steer a system between
assigned initial and final states, such that a trajectory-dependent cost
function is minimized. The application of optimal control to stochastic systems
is an open and challenging research frontier, with a spectrum of applications
ranging from stochastic thermodynamics to biophysics and data science. Among
these, the design of nanoscale electronic components motivates the study of
underdamped dynamics.
  In this work, we develop analytic techniques to determine protocols steering
finite time transitions at a minimum thermodynamic cost for stochastic
underdamped dynamics. As cost functions, we consider two paradigmatic
thermodynamic indicators. The first is the Kullback-Leibler divergence between
the probability measure of the controlled process and that of a reference
process. The corresponding optimization problem is the underdamped version of
the Schr\"odinger diffusion problem that has been widely studied in the
overdamped regime. The second is the mean entropy production during the
transition, corresponding to the second law of modern stochastic
thermodynamics. For transitions between Gaussian states, we show that optimal
protocols satisfy a Lyapunov equation, a central tool in stability analysis of
dynamical systems. For transitions between states described by general
Maxwell-Boltzmann distributions, we introduce an infinite-dimensional version
of the Poincar\'e-Linstedt multiscale perturbation theory around the overdamped
limit. This technique fundamentally improves the standard multiscale expansion.
Indeed, it enables the explicit computation of momentum cumulants, whose
variation in time is a distinctive trait of underdamped dynamics and is
directly accessible to experimental observation.
