  We establish an analogue of the classical Polya-Vinogradov inequality for
$GL(2, \F_p)$, where $p$ is a prime. In the process, we compute the `singular'
Gauss sums for $GL(2, \F_p)$. As an application, we show that the collection of
elements in $GL(2,\Z)$ whose reduction modulo $p$ are of maximal order in
$GL(2, \F_p)$ and whose matrix entries are bounded by $x$ has the expected size
as soon as $x\gg p^{1/2+\ep}$ for any $\ep>0$. In particular, there exist
elements in $GL(2,\Z)$ with matrix entries that are of the order
$O(p^{1/2+\ep})$ whose reduction modulo $p$ are primitive elements.
