  The 3D geometry of high-redshift galaxies remains poorly understood. We build
a differentiable Bayesian model and use Hamiltonian Monte Carlo to efficiently
and robustly infer the 3D shapes of star-forming galaxies in JWST-CEERS
observations with $\log M_*/M_{\odot}=9.0-10.5$ at $z=0.5-8.0$. We reproduce
previous results from HST-CANDELS in a fraction of the computing time and
constrain the mean ellipticity, triaxiality, size and covariances with samples
as small as $\sim50$ galaxies. We find high 3D ellipticities for all
mass-redshift bins suggesting oblate (disky) or prolate (elongated) geometries.
We break that degeneracy by constraining the mean triaxiality to be $\sim1$ for
$\log M_*/M_{\odot}=9.0-9.5$ dwarfs at $z>1$ (favoring the prolate scenario),
with significantly lower triaxialities for higher masses and lower redshifts
indicating the emergence of disks. The prolate population traces out a
``banana'' in the projected $b/a-\log a$ diagram with an excess of low $b/a$,
large $\log a$ galaxies. The dwarf prolate fraction rises from $\sim25\%$ at
$z=0.5-1.0$ to $\sim50-80\%$ at $z=3-8$. If these are disks, they cannot be
axisymmetric but instead must be unusually oval (triaxial) unlike local
circular disks. We simultaneously constrain the 3D size-mass relation and its
dependence on 3D geometry. High-probability prolate and oblate candidates show
remarkably similar S\'ersic indices ($n\sim1$), non-parametric morphological
properties and specific star formation rates. Both tend to be visually
classified as disks or irregular but edge-on oblate candidates show more dust
attenuation. We discuss selection effects, follow-up prospects and theoretical
implications.
