  The $\mathrm{SU}(r)$ Vafa-Witten partition function, which virtually counts
Higgs pairs on a projective surface $S$, was mathematically defined by
Tanaka-Thomas. On the Langlands dual side, the first-named author recently
introduced virtual counts of Higgs pairs on $\mu_r$-gerbes. In this paper, we
instead use Yoshioka's moduli spaces of twisted sheaves. Using Chern character
twisted by rational $B$-field, we give a new mathematical definition of the
$\mathrm{SU}(r) / \mathbb{Z}_r$ Vafa-Witten partition function when $r$ is
prime. Our definition uses the period-index theorem of de Jong.
  $S$-duality, a concept from physics, predicts that the $\mathrm{SU}(r)$ and
$\mathrm{SU}(r) / \mathbb{Z}_r$ partitions functions are related by a modular
transformation. We turn this into a mathematical conjecture, which we prove for
all $K3$ surfaces and prime numbers $r$.
