  The recent study in AdS$_3$/CFT$_2$ correspondence shows that the tree level
contribution and 1-loop correction of holographic R\'enyi entanglement entropy
(HRE) exactly match the direct CFT computation in the large central charge
limit. This allows the R\'enyi entanglement entropy to be a new window to study
the AdS/CFT correspondence. In this paper we generalize the study of R\'enyi
entanglement entropy in pure AdS$_3$ gravity to the massive gravity theories at
the critical points. For the cosmological topological massive gravity (CTMG),
the dual conformal field theory (CFT) could be a chiral conformal field theory
or a logarithmic conformal field theory (LCFT), depending on the asymptotic
boundary conditions imposed. In both cases, by studying the short interval
expansion of the R\'enyi entanglement entropy of two disjoint intervals with
small cross ratio $x$, we find that the classical and 1-loop HRE are in exact
match with the CFT results, up to order $x^6$. To this order, the difference
between the massless graviton and logarithmic mode can be seen clearly.
Moreover, for the cosmological new massive gravity (CNMG) at critical point,
which could be dual to a logarithmic CFT as well, we find the similar agreement
in the CNMG/LCFT correspondence. Furthermore we read the 2-loop correction of
graviton and logarithmic mode to HRE from CFT computation. It has distinct
feature from the one in pure AdS$_3$ gravity.
