  This paper considers dynamic transmit covariance design in point-to-point
MIMO fading systems with unknown channel state distributions and inaccurate
channel state information subject to both long term and short term power
constraints. First, the case of instantaneous but possibly inaccurate channel
state information at the transmitter (CSIT) is treated. By extending the
drift-plus-penalty technique, a dynamic transmit covariance policy is developed
and is shown to approach optimality with an $O(\delta)$ gap, where $\delta$ is
the inaccuracy measure of CSIT, regardless of the channel state distribution
and without requiring knowledge of this distribution. Next, the case of delayed
and inaccurate channel state information is considered. The optimal transmit
covariance solution that maximizes the ergodic capacity is fundamentally
different in this case, and a different online algorithm based on convex
projections is developed. The proposed algorithm for this delayed-CSIT case
also has an $O(\delta)$ optimality gap, where $\delta$ is again the inaccuracy
measure of CSIT.
