  We consider ergodic $\mathrm{Sym}(\mathbb{N})$-invariant probability measures
on the space of $L$-structures with domain $\mathbb{N}$ (for $L$ a countable
relational language), and call such a measure a properly ergodic structure when
no isomorphism class of structures is assigned measure $1$. We characterize
those theories in countable fragments of $\mathcal{L}_{\omega_1, \omega}$ for
which there is a properly ergodic structure concentrated on the models of the
theory. We show that for a countable fragment $F$ of $\mathcal{L}_{\omega_1,
\omega}$ the almost-sure $F$-theory of a properly ergodic structure has
continuum-many models (an analogue of Vaught's Conjecture in this context), but
its full almost-sure $\mathcal{L}_{\omega_1, \omega}$-theory has no models. We
also show that, for an $F$-theory $T$, if there is some properly ergodic
structure that concentrates on the class of models of $T$, then there are
continuum-many such properly ergodic structures.
