  We develop a systematic procedure to quantise canonically Hamiltonians of
light-matter models of transmission lines coupled through lumped linear
lossless ideal nonreciprocal elements, that break time-reversal symmetry, in a
circuit QED set-up. This is achieved through a description of the distributed
subsystems in terms of both flux and charge fields. We prove that this apparent
redundancy is required for the general derivation of the Hamiltonian for a
wider class of networks. By making use of the electromagnetic duality symmetry
in transmission lines (waveguides), we provide unambiguous identification of
the physical degrees of freedom, separating out the nondynamical parts. This
doubled description can also treat the case of other extended lumped
interactions in a regular manner that presents no spurious divergences, as we
show explicitly in the example of a circulator connected to a Josephson
junction through a transmission line. This theory enhances the quantum
engineering toolbox to design complex networks with nonreciprocal elements.
