  We define $G$-cospectrality of two $G$-gain graphs $(\Gamma,\psi)$ and
$(\Gamma',\psi')$, proving that it is a switching isomorphism invariant. When
$G$ is a finite group, we prove that $G$-cospectrality is equivalent to
cospectrality with respect to all unitary representations of $G$. Moreover, we
show that two connected gain graphs are switching equivalent if and only if the
gains of their closed walks centered at an arbitrary vertex $v$ can be
simultaneously conjugated. In particular, the number of switching equivalence
classes on an underlying graph $\Gamma$ with $n$ vertices and $m$ edges, is
equal to the number of simultaneous conjugacy classes of the group $G^{m-n+1}$.
We provide examples of $G$-cospectral non-switching isomorphic graphs and we
prove that any gain graph on a cycle is determined by its $G$-spectrum.
Moreover, we show that when $G$ is a finite cyclic group, the cospectrality
with respect to a faithful irreducible representation implies the cospectrality
with respect to any other faithful irreducible representation, and that the
same assertion is false in general.
