  Context. Fast Radio Bursts (FRBs) are millisecond-long bursts uniquely
detected at radio frequencies. FRB 131104 is the only case for which a
$\gamma$-ray transient positionally and temporally consistent was claimed. This
high-energy transient had a duration of $\sim400$~s and a 15-150~keV fluence
$S_{\gamma}\sim4\times10^{-6}$ erg $\mathrm{cm}^{-2}$. However, the association
with the FRB is still debated. Aims. We aim at testing the systematic presence
of an associated transient high-energy counterpart throughout a sample of the
FRB population. Methods. We used an approach like that used in machine learning
methodologies to accurately model the highly-variable Fermi/GBM instrumental
background on a time interval comparable to the duration of the proposed
$\gamma$-ray counterpart of FRB 131104. A possible $\gamma$-ray signal is then
constrained considering sample average lightcurves. Results. We constrain the
fluence of the possible $\gamma$-ray signal in the 8-1000 keV band down to $6.4
\times 10^{-7}$ ($7.1 \times 10^{-8}$) erg cm$^{-2}$ for a 200-s (1-s)
integration time. Furthermore, we found the radio-to-gamma fluence ratio to be
$\eta>10^{8}$ Jy ms erg$^{-1}$ cm$^2$. Conclusions. Our fluence limits exclude
$\sim 94\%$ of Fermi/GBM detected long gamma-ray bursts and $\sim 96\%$ of
Fermi/GBM detected short gamma-ray bursts. In addition, our limits on the
radio-to-gamma fluence ratio point to a different emission mechanism from that
of magnetar giant flares. Finally, we exclude a $\gamma$-ray counterpart as
fluent as the one possibly associated with FRB 131104 to be a common feature of
FRBs.
