  We present methods for locally solving the Dynamic Programming Equations
(DPE) and the Hamilton Jacobi Bellman (HJB) PDE that arise in the infinite
horizon optimal control problem. The method for solving the DPE is the discrete
time version of Al'brecht's procedure for locally approximating the solution of
the HJB. We also prove the existence of the smooth solutions to the DPE that
has the same Taylor series expansions as the formal solutions. Our procedure
for solving the HJB PDE numerically begins with Al'brecht's local solution as
the initial approximation and uses some Lyapunov criteria to piece together
polynomial estimates. The polynomials are generated using the method in the
Cauchy-Kovalevskaya Theorem.
