  This paper presents new sufficient conditions for convergence and asymptotic
or exponential stability of a stochastic discrete-time system, under which the
constructed Lyapunov function always decreases in expectation along the
system's solutions after a finite number of steps, but without necessarily
strict decrease at every step, in contrast to the classical stochastic Lyapunov
theory. As the first application of this new Lyapunov criterion, we look at the
product of any random sequence of stochastic matrices, including those with
zero diagonal entries, and obtain sufficient conditions to ensure the product
almost surely converges to a matrix with identical rows; we also show that the
rate of convergence can be exponential under additional conditions. As the
second application, we study a distributed network algorithm for solving linear
algebraic equations. We relax existing conditions on the network structures,
while still guaranteeing the equations are solved asymptotically.
