  We study inclusive and diffractive dijet production in electron-proton and
electron-nucleus collisions within the Color Glass Condensate effective field
theory. We compute dijet cross sections differentially in both mean dijet
transverse momentum $\mathbf{P}$ and recoil momentum $\mathbf{\Delta}$, as well
as the anisotropy in the relative angle between $\mathbf{P}$ and
$\mathbf{\Delta}$. We use the nonlinear Gaussian approximation to compute
multiparticle correlators for general small $x$ kinematics, employing running
coupling Balitsky-Kovchegov evolution to determine the dipole amplitude at
small $x$. Our results cover a much larger kinematic range than accessible in
previous computations performed in the correlation limit approximation, where
it is assumed that $|\mathbf{P}| \gg |\mathbf{\Delta}|$. We validate this
approximation in its range of applicability and quantify its failure for
$|\mathbf{P}| \lesssim |\mathbf{\Delta}|$. We also predict significant
target-dependent deviations from the correlation limit approximation for
$|\mathbf{P}| > |\mathbf{\Delta}|$ and $|\mathbf{P}| \lesssim Q_s$, which
offers a straightforward test of gluon saturation and access to multi-gluon
distributions at a future electron ion collider.
