  Value iteration (VI) is a ubiquitous algorithm for optimal control, planning,
and reinforcement learning schemes. Under the right assumptions, VI is a vital
tool to generate inputs with desirable properties for the controlled system,
like optimality and Lyapunov stability. As VI usually requires an infinite
number of iterations to solve general nonlinear optimal control problems, a key
question is when to terminate the algorithm to produce a "good" solution, with
a measurable impact on optimality and stability guarantees. By carefully
analysing VI under general stabilizability and detectability properties, we
provide explicit and novel relationships of the stopping criterion's impact on
near-optimality, stability and performance, thus allowing to tune these
desirable properties against the induced computational cost. The considered
class of stopping criteria encompasses those encountered in the control,
dynamic programming and reinforcement learning literature and it allows
considering new ones, which may be useful to further reduce the computational
cost while endowing and satisfying stability and near-optimality properties. We
therefore lay a foundation to endow machine learning schemes based on VI with
stability and performance guarantees, while reducing computational complexity.
