  We study the evolution of the quantum state of $n$ photons in $m$ different
modes when they go through a lossless linear optical system. We show that there
are quantum evolution operators $U$ that cannot be built with linear optics
alone unless the number of photons or the number of modes is equal to one. The
evolution for single photons can be controlled with the known realization of
any unitary proved by Reck, Zeilinger, Bernstein and Bertani. The evolution for
a single mode corresponds to the trivial evolution in a phase shifter. We
analyze these two cases and prove that any other combination of the number of
photons and modes produces a Hilbert state too big for the linear optics system
to give any desired evolution.
