  We develop a theoretical framework to characterize the decoherence dynamics
due to multi-photon scattering in an all-optical switch based on Rydberg atom
induced nonlinearities. By incorporating the knowledge of this decoherence
process into optimal photon storage and retrieval strategies, we establish
optimised switching protocols for experimentally relevant conditions, and
evaluate the corresponding limits in the achievable fidelities. Based on these
results we work out a simplified description that reproduces recent experiments
[arXiv:1511.09445] and provides a new interpretation in terms of many-body
decoherence involving multiple incident photons and multiple gate excitations
forming the switch. Aside from offering insights into the operational capacity
of realistic photon switching capabilities, our work provides a complete
description of spin wave decoherence in a Rydberg quantum optics setting, and
has immediate relevance to a number of further applications employing photon
storage in Rydberg media.
