  We propose a numerical accountant for evaluating the tight
$(\varepsilon,\delta)$-privacy loss for algorithms with discrete one
dimensional output. The method is based on the privacy loss distribution
formalism and it uses the recently introduced fast Fourier transform based
accounting technique. We carry out an error analysis of the method in terms of
moment bounds of the privacy loss distribution which leads to rigorous lower
and upper bounds for the true $(\varepsilon,\delta)$-values. As an application,
we present a novel approach to accurate privacy accounting of the subsampled
Gaussian mechanism. This completes the previously proposed analysis by giving
strict lower and upper bounds for the privacy parameters. We demonstrate the
performance of the accountant on the binomial mechanism and show that our
approach allows decreasing noise variance up to 75 percent at equal privacy
compared to existing bounds in the literature. We also illustrate how to
compute tight bounds for the exponential mechanism applied to counting queries.
