  We continue the analysis of modular invariant functions, subject to
inhomogeneous Laplace eigenvalue equations, that were determined in terms of
Poincar\'e series in a companion paper. The source term of the Laplace equation
is a product of (derivatives of) two non-holomorphic Eisenstein series whence
the modular invariants are assigned depth two. These modular invariant
functions can sometimes be expressed in terms of single-valued iterated
integrals of holomorphic Eisenstein series as they appear in generating series
of modular graph forms. We show that the set of iterated integrals of
Eisenstein series has to be extended to include also iterated integrals of
holomorphic cusp forms to find expressions for all modular invariant functions
of depth two. The coefficients of these cusp forms are identified as ratios of
their L-values inside and outside the critical strip.
