  Time-frequency (TF) filtering of analog signals has played a crucial role in
the development of radio-frequency communications, and is currently being
recognized as an essential capability for communications, both classical and
quantum, in the optical frequency domain. How best to design optical
time-frequency (TF) filters to pass a targeted temporal mode (TM), and to
reject background (noise) photons in the TF detection window? The solution for
coherent TF filtering is known, the quantum pulse gate, whereas the
conventional, more common method is implemented by a sequence of incoherent
spectral filtering and temporal gating operations. To compare these two
methods, we derive a general formalism for two-stage incoherent time frequency
filtering, finding expressions for signal pulse transmission efficiency, and
for the ability to discriminate TMs, which allows the blocking of unwanted
background light. We derive the tradeoff between efficiency and TM
discrimination ability, and find a remarkably concise relation between these
two quantities and the time-bandwidth product of the combined filters. We apply
the formalism to two examples, rectangular filters or Gaussian filters, both of
which have known orthogonal-function decompositions. The formalism can be
applied to any state of light occupying the input temporal mode, e.g.,
classical coherent-state signals or pulsed single photon states of light. We
point out implications in classical and quantum optical communications. As an
example, we study quantum key distribution, wherein strong rejection of
background noise is necessary to maintain a high quality of entanglement, while
high signal transmission is needed to ensure a useful key generation rate.
