  \textit{Ab initio} calculations of electron-phonon interactions including the
polar Fr\"ohlich coupling have advanced considerably in recent years. The
Fr\"ohlich electron-phonon matrix element is by now well understood in the case
of bulk three-dimensional (3D) materials. In the case of two-dimensional (2D)
materials, the standard procedure to include Fr\"ohlich coupling is to employ
Coulomb truncation, so as to eliminate artificial interactions between periodic
images of the 2D layer. While these techniques are well established, the
transition of the Fr\"ohlich coupling from three to two dimensions has not been
investigated. Furthermore, it remains unclear what error one makes when
describing 2D systems using the standard bulk formalism in a periodic supercell
geometry. Here, we generalize previous work on the \textit{ab initio}
Fr\"ohlich electron-phonon matrix element in bulk materials by investigating
the electrostatic potential of atomic dipoles in a periodic supercell
consisting of a 2D material and a continuum dielectric slab. We obtain a
unified expression for the matrix element, which reduces to the existing
formulas for 3D and 2D systems when the interlayer separation tends to zero or
infinity, respectively. This new expression enables an accurate description of
the Fr\"ohlich matrix element in 2D systems without resorting to Coulomb
truncation. We validate our approach by direct \textit{ab initio}
density-functional perturbation theory calculations for monolayer BN and
MoS$_2$, and we provide a simple expression for the 2D Fr\"ohlich matrix
element that can be used in model Hamiltonian approaches. The formalism
outlined in this work may find applications in calculations of polarons,
quasiparticle renormalization, transport coefficients, and superconductivity,
in 2D and quasi-2D materials.
