  The action of a finite group $G$ on a compact Riemann surface $X$ naturally
induces another action of $G$ on its Jacobian variety $\operatorname{J}(X)$. In
many cases, each component of the group algebra decomposition of
$\operatorname{J}(X)$ is isogenous to a Prym varieties of an intermediate
covering of the Galois covering $\pi_G\colon X \to X/G$; in such a case, we say
that the group algebra decomposition is affordable by Prym varieties. In this
article, we present an infinite family of groups that act on Riemann surfaces
in a manner that the group algebra decomposition of $\operatorname{J}(X)$ is
not affordable by Prym varieties; namely, affine groups
$\operatorname{Aff}(\mathbb{F}_q)$ with some exceptions: $q = 2$, $q = 9$, $q$
a Fermat prime, $q = 2^n$ with $2^n-1$ a Mersenne prime and some particular
cases when $X/G$ has genus $0$ or $1$. In each one of this exceptional cases,
we give the group algebra decomposition of $\operatorname{J}(X)$ by Prym
varieties.
