  Many applications in computational sciences and statistical inference require
the computation of expectations with respect to complex high-dimensional
distributions with unknown normalization constants, as well as the estimation
of these constants. Here we develop a method to perform these calculations
based on generating samples from a simple base distribution, transporting them
by the flow generated by a velocity field, and performing averages along these
flowlines. This non-equilibrium importance sampling (NEIS) strategy is
straightforward to implement and can be used for calculations with arbitrary
target distributions. On the theory side, we discuss how to tailor the velocity
field to the target and establish general conditions under which the proposed
estimator is a perfect estimator with zero-variance. We also draw connections
between NEIS and approaches based on mapping a base distribution onto a target
via a transport map. On the computational side, we show how to use deep
learning to represent the velocity field by a neural network and train it
towards the zero variance optimum. These results are illustrated numerically on
benchmark examples (with dimension up to $10$), where after training the
velocity field, the variance of the NEIS estimator is reduced by up to $6$
orders of magnitude than that of a vanilla estimator. We also compare the
performances of NEIS with those of Neal's annealed importance sampling (AIS).
