  The studies of the Higgs boson couplings based on the recent and upcoming LHC
data open up a new window on physics beyond the Standard Model. In this paper,
we propose a statistical guide to the consistent treatment of the theoretical
uncertainties entering the Higgs rate fits. Both the Bayesian and frequentist
approaches are systematically analysed in a unified formalism. We present
analytical expressions for the marginal likelihoods, useful to implement
simultaneously the experimental and theoretical uncertainties. We review the
various origins of the theoretical errors (QCD, EFT, PDF, production mode
contamination...). All these individual uncertainties are thoroughly combined
with the help of moment-based considerations. The theoretical correlations
among Higgs detection channels appear to affect the location and size of the
best-fit regions in the space of Higgs couplings. We discuss the recurrent
question of the shape of the prior distributions for the individual theoretical
errors and find that a nearly Gaussian prior arises from the error
combinations. We also develop the bias approach, which is an alternative to
marginalisation providing more conservative results. The statistical framework
to apply the bias principle is introduced and two realisations of the bias are
proposed. Finally, depending on the statistical treatment, the Standard Model
prediction for the Higgs signal strengths is found to lie within either the
$68\%$ or $95\%$ confidence level region obtained from the latest analyses of
the $7$ and $8$ TeV LHC datasets.
