  In this paper, we consider the delay-constrained beamforming control for
downlink multi-user MIMO (MU- MIMO) systems with imperfect channel state
information at the transmitter (CSIT). The delay-constrained control problem is
formulated as an infinite horizon average cost partially observed Markov
decision process. To deal with the curse of dimensionality, we introduce a
virtual continuous time system and derive a closed-form approximate value
function using perturbation analysis w.r.t. the CSIT errors. To deal with the
challenge of the conditional packet error rate (PER), we build a tractable
closed- form approximation using a Bernstein-type inequality. Based on the
closed-form approximations of the relative value function and the conditional
PER, we propose a conservative formulation of the original beamforming control
problem. The conservative problem is non-convex and we transform it into a
convex problem using the semidefinite relaxation (SDR) technique. We then
propose an alternating iterative algorithm to solve the SDR problem. Finally,
the proposed scheme is compared with various baselines through simulations and
it is shown that significant performance gain can be achieved.
