  In this article we present a generalised view on Path Integral Control (PIC)
methods. PIC refers to a particular class of policy search methods that are
closely tied to the setting of Linearly Solvable Optimal Control (LSOC), a
restricted subclass of nonlinear Stochastic Optimal Control (SOC) problems.
This class is unique in the sense that it can be solved explicitly to yield a
formal optimal state trajectory distribution. In this contribution we first
review the PIC theory and discuss related algorithms tailored to policy search
in general. We are able to identify a generic design strategy that relies on
the existence of an optimal state trajectory distribution and finds a
parametric policy by minimizing the cross entropy between the optimal and a
state trajectory distribution parametrized through its policy. Inspired by this
observation we then aim to formulate a SOC problem that shares traits with the
LSOC setting yet that covers a less restrictive class of problem formulations.
We refer to this SOC problem as Entropy Regularized Trajectory Optimization.
The problem is closely related to the Entropy Regularized Stochastic Optimal
Control setting which is lately often addressed by the Reinforcement Learning
(RL) community. We analyse the theoretical convergence behaviour of the
theoretical state trajectory distribution sequence and draw connections with
stochastic search methods tailored to classic optimization problems. Finally we
derive explicit updates and compare the implied Entropy Regularized PIC with
earlier work in the context of both PIC and RL for derivative-free trajectory
optimization.
