  Holographic Metasurface Transceivers (HMTs) are emerging as cost-effective
substitutes to large antenna arrays for beamforming in Millimeter and TeraHertz
wave communication. However, to achieve desired channel gains through
beamforming in HMT, phase-shifts of a large number of elements need to be
appropriately set, which is challenging. Also, these optimal phase-shifts
depend on the location of the receivers, which could be unknown. In this work,
we develop a learning algorithm using a {\it fixed-budget multi-armed bandit
framework} to beamform and maximize received signal strength at the receiver
for far-field regions. Our algorithm, named \Algo exploits the parametric form
of channel gains of the beams, which can be expressed in terms of two {\it
phase-shifting parameters}. Even after parameterization, the problem is still
challenging as phase-shifting parameters take continuous values. To overcome
this, {\it\HB} works with the discrete values of phase-shifting parameters and
exploits their unimodal relations with channel gains to learn the optimal
values faster. We upper bound the probability of {\it\HB} incorrectly
identifying the (discrete) optimal phase-shift parameters in terms of the
number of pilots used in learning. We show that this probability decays
exponentially with the number of pilot signals. We demonstrate that {\it\HB}
outperforms state-of-the-art algorithms through extensive simulations.
