  Dust is a key component of galaxies, but its properties during the earliest
eras of structure formation remain elusive. Here we present a simple
semi-analytic model of the dust distribution in galaxies at $z \gtrsim 5$. We
calibrate the free parameters of this model to estimates of the UV attenuation
(using the IRX-$\beta$ relation between infrared emission and the UV spectral
slope) and to ALMA measurements of dust emission. We find that the observed
dust emission requires that most of the dust expected in these galaxies is
retained (assuming a similar yield to lower-redshift sources), but if the dust
is spherically distributed, the modest attenuation requires that it be
significantly more extended than the stars. Interestingly, the retention
fraction is larger for less massive galaxies in our model. However, the
required radius is a significant fraction of the host's virial radius and is
larger than the estimated extent of dust emission from stacked high-$z$
galaxies. These can be reconciled if the dust is distributed anisotropically,
with typical covering fractions of $\sim 0.2$--0.7 in bright galaxies and
$\lesssim 0.1$ in fainter ones.
