  In this paper, we study the almost sure boundedness and the convergence of
the stochastic approximation (SA) algorithm. At present, most available
convergence proofs are based on the ODE method, and the almost sure boundedness
of the iterations is an assumption and not a conclusion. In Borkar-Meyn (2000),
it is shown that if the ODE has only one globally attractive equilibrium, then
under additional assumptions, the iterations are bounded almost surely, and the
SA algorithm converges to the desired solution. Our objective in the present
paper is to provide an alternate proof of the above, based on martingale
methods, which are simpler and less technical than those based on the ODE
method. As a prelude, we prove a new sufficient condition for the global
asymptotic stability of an ODE. Next we prove a "converse" Lyapunov theorem on
the existence of a suitable Lyapunov function with a globally bounded Hessian,
for a globally exponentially stable system. Both theorems are of independent
interest to researchers in stability theory. Then, using these results, we
provide sufficient conditions for the almost sure boundedness and the
convergence of the SA algorithm. We show through examples that our theory
covers some situations that are not covered by currently known results,
specifically Borkar-Meyn (2000).
