  In this paper, we study the problem of dynamic channel allocation for URLLC
traffic in a multi-user multi-channel wireless network where urgent packets
have to be successfully transmitted in a timely manner. We formulate the
problem as a finite-horizon Markov Decision Process with a stochastic
constraint related to the QoS requirement, defined as the packet loss rate for
each user. We propose a novel weighted formulation that takes into account both
the total expected reward (number of successfully transmitted packets) and the
risk which we define as the QoS requirement violation. First, we use the value
iteration algorithm to find the optimal policy, which assumes a perfect
knowledge of the controller of all the parameters, namely the channel
statistics. We then propose a Q-learning algorithm where the controller learns
the optimal policy without having knowledge of neither the CSI nor the channel
statistics. We illustrate the performance of our algorithms with numerical
studies.
