  We derive an explicit formula for the connected $(n,m)$-point functions
associated to an arbitrary diagonal tau-function
$\tau_f(\boldsymbol{t}^+,\boldsymbol{t}^-)$ of the 2d Toda lattice hierarchy
using fermionic computations and the boson-fermion correspondence. Then for
fixed $\boldsymbol{t}^-$, we compute the KP-affine coordinates of
$\tau_f(\boldsymbol{t}^+,\boldsymbol{t}^-)$. As applications, we present a
unified approach to compute various types of connected double Hurwitz numbers,
including the ordinary double Hurwitz numbers, the double Hurwitz numbers with
completed $r$-cycles, and the mixed double Hurwitz numbers. We also apply this
method to the computation of the stationary Gromov-Witten invariants of
$\mathbb P^1$ relative to two points.
