  The definitions of the $n^{th}$ Gauss sum and the associated $n^{th}$ central
charge are introduced for premodular categories $\mathcal{C}$ and
$n\in\mathbb{Z}$. We first derive an expression of the $n^{th}$ Gauss sum of a
modular category $\mathcal{C}$, for any integer $n$ coprime to the order of the
T-matrix of $\mathcal{C}$, in terms of the first Gauss sum, the global
dimension, the twist and their Galois conjugates. As a consequence, we show for
these $n$, the higher Gauss sums are $d$-numbers and the associated central
charges are roots of unity. In particular, if $\mathcal{C}$ is the Drinfeld
center of a spherical fusion category, then these higher central charges are 1.
We obtain another expression of higher Gauss sums for de-equivariantization and
local module constructions of appropriate premodular and modular categories.
These expressions are then applied to prove the Witt invariance of higher
central charges for pseudounitary modular categories.
