  The recurrent geometric network (RGN), the first end-to-end differentiable
neural architecture for protein structure prediction, is a competitive
alternative to existing models. However, the RGN's use of recurrent neural
networks (RNNs) as internal representations results in long training time and
unstable gradients. And because of its sequential nature, it is less effective
at learning global dependencies among amino acids than existing transformer
architectures. We propose the Universal Transforming Geometric Network (UTGN),
an end-to-end differentiable model that uses the encoder portion of the
Universal Transformer architecture as an alternative for internal
representations. Our experiments show that compared to RGN, UTGN achieve a
$1.7$ \si{\angstrom} improvement on the free modeling portion and a $0.7$
\si{\angstrom} improvement on the template based modeling of the CASP12
competition.
