  Vafa-Witten (VW) theory is a topologically twisted version of N=4
supersymmetric Yang-Mills theory. S-duality suggests that the partition
function of VW theory with gauge group SU(N) transforms as a modular form under
duality transformations. Interestingly, Vafa and Witten demonstrated the
presence of a modular anomaly, when the theory has gauge group SU(2) and is
considered on the complex projective plane P2. This modular anomaly could be
expressed as an integral of a modular form, and also be traded for a
holomorphic anomaly. We demonstrate that the modular anomaly for gauge group
SU(3) involves an iterated integral of modular forms. Moreover, the modular
anomaly for SU(3) can be traded for a holomorphic anomaly, which is shown to
factor into a product of the partition functions for lower rank gauge groups.
The SU(3) partition function is mathematically an example of a mock modular
form of depth two.
