  Non-Markovian time-evolution of quantum systems is a challenging problem,
often mitigated by employing numerical methods or making simplifying
assumptions. In this work, we address this problem in waveguide QED by
developing a diagrammatic approach, which performs fully analytical
non-Markovian time evolution of single-photon states. By revisiting Fermi's two
atom problem, we tackle the impeding question of whether rotating-wave
approximation violates causality in single-photon waveguide QED. Afterward, we
introduce and prove the \emph{no upper half-plane poles (no-UHP) theorem},
which connects the poles of scattering parameters to the causality principle.
Finally, we visualize the time-delayed coherent quantum feedback mediated by
the field, discuss the Markovian limit for microscopically separated qubits
where short-distance causality violations occur and the emergence of collective
decay rates in this limit. Our diagrammatic approach is the first method to
perform exact and analytical non-Markovian time evolution of multi-emitter
systems in waveguide QED.
