  The amount of information lost in sub-Nyquist sampling of a continuous-time
Gaussian stationary process is quantified. We consider a combined source coding
and sub-Nyquist reconstruction problem in which the input to the encoder is a
noisy sub-Nyquist sampled version of the analog source. We first derive an
expression for the mean squared error in the reconstruction of the process from
a noisy and information rate-limited version of its samples. This expression is
a function of the sampling frequency and the average number of bits describing
each sample. It is given as the sum of two terms: Minimum mean square error in
estimating the source from its noisy but otherwise fully observed sub-Nyquist
samples, and a second term obtained by reverse waterfilling over an average of
spectral densities associated with the polyphase components of the source. We
extend this result to multi-branch uniform sampling, where the samples are
available through a set of parallel channels with a uniform sampler and a
pre-sampling filter in each branch. Further optimization to reduce distortion
is then performed over the pre-sampling filters, and an optimal set of
pre-sampling filters associated with the statistics of the input signal and the
sampling frequency is found. This results in an expression for the minimal
possible distortion achievable under any analog to digital conversion scheme
involving uniform sampling and linear filtering. These results thus unify the
Shannon-Whittaker-Kotelnikov sampling theorem and Shannon rate-distortion
theory for Gaussian sources.
