  Independent learning (IL), despite being a popular approach in practice to
achieve scalability in large-scale multi-agent systems, usually lacks global
convergence guarantees. In this paper, we study two representative algorithms,
independent $Q$-learning and independent natural actor-critic, within
value-based and policy-based frameworks, and provide the first finite-sample
analysis for approximate global convergence. The results imply a sample
complexity of $\tilde{\mathcal{O}}(\epsilon^{-2})$ up to an error term that
captures the dependence among agents and characterizes the fundamental limit of
IL in achieving global convergence. To establish the result, we develop a novel
approach for analyzing IL by constructing a separable Markov decision process
(MDP) for convergence analysis and then bounding the gap due to model
difference between the separable MDP and the original one. Moreover, we conduct
numerical experiments using a synthetic MDP and an electric vehicle charging
example to verify our theoretical findings and to demonstrate the practical
applicability of IL.
