  Full-wave simulations are indispensable for nanophotonics and
electromagnetics but are severely constrained on large systems, especially
multi-channel ones such as disordered media, aperiodic metasurfaces, and
densely packed photonic circuits where each input requires a large-scale
simulation. Here we bypass the computationally demanding solution of Maxwell's
equations and directly evaluate the full-wave multi-input response, with no
approximation. We augment the Maxwell operator with all input source profiles
and output projection profiles, followed by a single partial factorization that
directly yields the entire multi-input scattering matrix via the Schur
complement. This method is simple to implement and applies to any linear
partial differential equation. Its advantage grows with size, being 1,000 to
30,000,000 times faster than existing methods for systems with about ten
million variables. We use it to realize the first full-wave simulations of
entangled-photon backscattering from disorder and all-angle characterizations
of high-numerical-aperture metalenses that are thousands of wavelengths wide.
This work reveals the significant efficiency gain when we rethink what to
compute and enables the exploration of diverse multi-channel systems.
