  In this paper, we study the outage minimization problem in a
decode-and-forward cooperative network with relay uncertainty. To reduce the
outage probability and improve the quality of service, existing researches
usually rely on the assumption of both exact instantaneous channel state
information (CSI) and environmental uncertainty. However, it is difficult to
obtain perfect instantaneous CSI immediately under practical situations where
channel states change rapidly, and the uncertainty in communication
environments may not be observed, which makes traditional methods not
applicable. Therefore, we turn to reinforcement learning (RL) methods for
solutions, which do not need any prior knowledge of underlying channel or
assumptions of environmental uncertainty. RL method is to learn from the
interaction with communication environment, optimize its action policy, and
then propose relay selection and power allocation schemes. We first analyse the
robustness of RL action policy by giving the lower bound of the worst-case
performance, when RL methods are applied to communication scenarios with
environment uncertainty. Then, we propose a robust algorithm for outage
probability minimization based on RL. Simulation results reveal that compared
with traditional RL methods, our approach has better generalization ability and
can improve the worst-case performance by about 6% when evaluated in unseen
environments.
