  Reinforcement learning is commonly used with function approximation. However,
very few positive results are known about the convergence of function
approximation based RL control algorithms. In this paper we show that TD(0) and
Sarsa(0) with linear function approximation is convergent for a simple class of
problems, where the system is linear and the costs are quadratic (the LQ
control problem). Furthermore, we show that for systems with Gaussian noise and
non-completely observable states (the LQG problem), the mentioned RL algorithms
are still convergent, if they are combined with Kalman filtering.
