  We investigate the mass structure of a strong gravitational lens galaxy at
$z=0.350$, taking advantage of the milli-arcsecond (mas) angular resolution of
very long baseline interferometric (VLBI) observations. In the first analysis
of its kind at this resolution, we jointly infer the lens model parameters and
pixellated radio source surface brightness. We consider several lens models of
increasing complexity, starting from an elliptical power-law density profile.
We extend this model to include angular multipole structures, a separate
stellar mass component, additional nearby field galaxies, and/or a generic
external potential. We compare these models using their relative Bayesian
log-evidence (Bayes factor). We find strong evidence for angular structure in
the lens; our best model is comprised of a power-law profile plus multipole
perturbations and external potential, with a Bayes factor of $+14984$ relative
to the elliptical power-law model. It is noteworthy that the elliptical
power-law mass distribution is a remarkably good fit on its own, with
additional model complexity correcting the deflection angles only at the
$\sim5$ mas level. We also consider the effects of added complexity in the lens
model on time-delay cosmography and flux-ratio analyses. We find that an overly
simplistic power-law ellipsoid lens model can bias the measurement of $H_0$ by
$\sim3$ per cent and mimic flux ratio anomalies of $\sim8$ per cent. Our
results demonstrate the power of high-resolution VLBI observations to provide
strong constraints on the inner density profiles of lens galaxies.
