  We consider a novel contribution to the polarization of the Cosmic Microwave
Background induced by vector and tensor modes generated by the non-linear
evolution of primordial scalar perturbations. Our calculation is based on
relativistic second-order perturbation theory and allows to estimate the
effects of these secondary modes on the polarization angular power-spectra. We
show that a non-vanishing B-mode polarization unavoidably arises from pure
scalar initial perturbations, thus limiting our ability to detect the signature
of primordial gravitational waves generated during inflation. This secondary
effect dominates over that of primordial tensors for an inflationary
tensor-to-scalar ratio $r<10^{-6}$. The magnitude of the effect is smaller than
the contamination produced by the conversion of polarization of type E into
type B, by weak gravitational lensing. However the lensing signal can be
cleaned, making the secondary modes discussed here the actual background
limiting the detection of small amplitude primordial gravitational waves.
