  We study the problem of calibrating a quantum receiver for optical coherent
states when transmitted on a quantum optical channel with variable
transmissivity, a common model for long-distance optical-fiber and
free/deep-space optical communication. We optimize the error probability of
legacy adaptive receivers, such as Kennedy's and Dolinar's, on average with
respect to the channel transmissivity distribution. We then compare our results
with the ultimate error probability attainable by a general quantum device,
computing the Helstrom bound for mixtures of coherent-state hypotheses, for the
first time to our knowledge, and with homodyne measurements. With these tools,
we first analyze the simplest case of two different transmissivity values; we
find that the strategies adopted by adaptive receivers exhibit strikingly new
features as the difference between the two transmissivities increases. Finally,
we employ a recently introduced library of shallow reinforcement learning
methods, demonstrating that an intelligent agent can learn the optimal receiver
setup from scratch by training on repeated communication episodes on the
channel with variable transmissivity and receiving rewards if the
coherent-state message is correctly identified.
