  The convergence properties of the upwind difference scheme for the
Hamilton-Jacobi-Bellman (HJB) equation, which is a fundamental equation for
optimal control theory, are investigated. We first perform a convergence
analysis for the solution of the scheme, which eliminates ambiguities in the
proofs of existing studies. We then prove the convergence of the spatial
difference of the solution in the scheme by the correspondence between the HJB
equations and the conservation laws. This result leads to a property of the
objective function called epi-convergence, by which the convergence property of
the input function is shown. The latter two results have not been addressed in
existing studies. Numerical calculations support the obtained results.
