  For any suitable Optimal Control Problem (OCP) there exists a value function,
defined as the unique viscosity solution to the Hamilton-Jacobi-Bellman (HJB)
Partial-Differential-Equation (PDE), and which can be used to design an optimal
feedback controller for the given OCP. In this paper, we approximately solve
the HJB-PDE by proposing a sequence of Sum-Of-Squares (SOS) problems, each of
which yields a polynomial subsolution to the HJB-PDE. We show that the
resulting sequence of polynomial sub-solutions converges to the value function
of the OCP in the L1 norm. Furthermore, for each polynomial sub-solution in
this sequence, we show that the associated sequence of sublevel sets converge
to the sublevel set of the value function of the OCP in the volume metric.
Next, for any approximate value function, obtained from an SOS program or any
other method (e.g. discretization), we construct an associated feedback
controller, and show that sub-optimality of this controller as applied to the
OCP is bounded by the distance between the approximate and true value function
of the OCP in the Sobolev norm. Finally, we demonstrate numerically that by
solving our proposed SOS problem we are able to accurately approximate value
functions, design controllers and estimate reachable sets.
