  In recent years, the integration of communication and control systems has
gained significant traction in various domains, ranging from autonomous
vehicles to industrial automation and beyond. Multi-armed bandit (MAB)
algorithms have proven their effectiveness as a robust framework for solving
control problems. In this work, we investigate the use of MAB algorithms to
control remote devices, which faces considerable challenges primarily
represented by latency and reliability. We analyze the effectiveness of MABs
operating in environments where the action feedback from controlled devices is
transmitted over an unreliable communication channel and stored in a Geo/Geo/1
queue. We investigate the impact of queue sampling strategies on the MAB
performance, and introduce a new stochastic approach. Its performance in terms
of regret is evaluated against established algorithms in the literature for
both upper confidence bound (UCB) and Thompson Sampling (TS) algorithms.
Additionally, we study the trade-off between maximizing rewards and minimizing
energy consumption.
