  We describe the automorphism groups of finite $p$-groups arising naturally
via Hessian determinantal representations of elliptic curves defined over
number fields. Moreover, we derive explicit formulas for the orders of these
automorphism groups for elliptic curves of $j$-invariant $1728$ given in
Weierstrass form. We interpret these orders in terms of the numbers of
$3$-torsion points (or flex points) of the relevant curves over finite fields.
Our work greatly generalizes and conceptualizes previous examples given by du
Sautoy and Vaughan-Lee. It explains, in particular, why the orders arising in
these examples are polynomial on Frobenius sets and vary with the primes in a
nonquasipolynomial manner.
