  The promising coverage and spectral efficiency gains of intelligent
reflecting surfaces (IRSs) are attracting increasing interest. In order to
realize these surfaces in practice, however, several challenges need to be
addressed. One of these main challenges is how to configure the reflecting
coefficients on these passive surfaces without requiring massive channel
estimation or beam training overhead. Earlier work suggested leveraging
supervised learning tools to design the IRS reflection matrices. While this
approach has the potential of reducing the beam training overhead, it requires
collecting large datasets for training the neural network models. In this
paper, we propose a novel deep reinforcement learning framework for predicting
the IRS reflection matrices with minimal training overhead. Simulation results
show that the proposed online learning framework can converge to the optimal
rate that assumes perfect channel knowledge. This represents an important step
towards realizing a standalone IRS operation, where the surface configures
itself without any control from the infrastructure.
