  We consider the two-dimensional (2d) Ising model on a infinitely long
cylinder and study the probabilities $p_i$ to observe a given spin
configuration $i$ along a circular section of the cylinder. These probabilities
also occur as eigenvalues of reduced density matrices in some Rokhsar-Kivelson
wave-functions. We analyze the subleading constant to the R\'enyi entropy
$R_n=1/(1-n) \ln (\sum_i p_i^n)$ and discuss its scaling properties at the
critical point. Studying three different microscopic realizations, we provide
numerical evidence that it is universal and behaves in a step-like fashion as a
function of $n$, with a discontinuity at the Shannon point $n=1$. As a
consequence, a field theoretical argument based on the replica trick would fail
to give the correct value at this point. We nevertheless compute it numerically
with high precision. Two other values of the R\'enyi parameter are of special
interest: $n=1/2$ and $n=\infty$ are related in a simple way to the
Affleck-Ludwig boundary entropies associated to free and fixed boundary
conditions respectively.
