  We present a fully passive method for implementing a quantum phase gate
between two photons travelling in a one-dimensional wave guide. The gate is
based on chirally coupled emitters in a three-level $V$ configuration, which
only interact through the photon field without any external control fields. We
describe the (non-)linear scattering of the emerging polariton states and show
that for near resonant photons the scattering dynamics directly implements a
perfect control phase gate between the incoming photons in the limit of many
emitters. For a finite number of emitters we show that the dominant error
mechanism can be suppressed by a simple frequency filter at the cost of a minor
reduction in the success probability. We verify the results via comparison with
exact scattering matrix theory and show that the fidelity can reach values
$\mathcal{F}\sim99\%$ with a gate success probability of $>99\%$ for as few as
8 emitters.
