  This paper proposes an online transfer framework to capture the interaction
among agents and shows that current transfer learning in reinforcement learning
is a special case of online transfer. Furthermore, this paper re-characterizes
existing agents-teaching-agents methods as online transfer and analyze one such
teaching method in three ways. First, the convergence of Q-learning and Sarsa
with tabular representation with a finite budget is proven. Second, the
convergence of Q-learning and Sarsa with linear function approximation is
established. Third, the we show the asymptotic performance cannot be hurt
through teaching. Additionally, all theoretical results are empirically
validated.
