  Let $G$ be a locally compact group and $\mu$ a probability measure on $G,$
which is not assumed to be absolutely continuous with respect to Haar measure.
Given a unitary representation $(\pi, \cal H)$ of $G,$ we study spectral
properties of the operator $\pi(\mu)$ acting on $\cal H.$ Assume that $\mu$ is
adapted and that the trivial representation $1_G$ is not weakly contained in
the tensor product $\pi\otimes \bar\pi.$ We show that $\pi(\mu)$ has a spectral
gap, that is, for the spectral radius $r_{\rm spec}(\pi(\mu))$ of $\pi(\mu),$
we have $r_{\rm spec}(\pi(\mu))<1.$ This provides a common generalization of
several previously known results. Another consequence is that, if $G$ has
Kazhdan's Property (T), then $r_{\rm spec}(\pi(\mu))<1$ for every unitary
representation $\pi$ of $G$ without finite dimensional subrepresentations.
Moreover, we give new examples of so-called identity excluding groups.
