  Recent string theory tests of swampland ideas like the distance or the dS
conjectures have been performed at weak coupling. Testing these ideas beyond
the weak coupling regime remains challenging. We propose to exploit the modular
symmetries of the moduli effective action to check swampland constraints beyond
perturbation theory. As an example we study the case of heterotic 4d
$\mathcal{N}=1$ compactifications, whose non-perturbative effective action is
known to be invariant under modular symmetries acting on the K\"ahler and
complex structure moduli, in particular $SL(2,Z)$ T-dualities (or subgroups
thereof) for 4d heterotic or orbifold compactifications. Remarkably, in models
with non-perturbative superpotentials, the corresponding duality invariant
potentials diverge at points at infinite distance in moduli space. The
divergence relates to towers of states becoming light, in agreement with the
distance conjecture. We discuss specific examples of this behavior based on
gaugino condensation in heterotic orbifolds. We show that these examples are
dual to compactifications of type I' or Horava-Witten theory, in which the
$SL(2,Z)$ acts on the complex structure of an underlying 2-torus, and the tower
of light states correspond to D0-branes or M-theory KK modes. The
non-perturbative examples explored point to potentials not leading to weak
coupling at infinite distance, but rather diverging in the asymptotic corners
of moduli space, dynamically forbidding the access to points with global
symmetries. We perform a study of general modular invariant potentials and find
that there are dS maxima and saddle points but no dS minima, and that all
examples explored obey the refined dS conjecture.
