  The physical sciences require models tailored to specific nuances of
different dynamics. In this work, we study outcome predictions in nuclear
fusion tokamaks, where a major challenge are \textit{disruptions}, or the loss
of plasma stability with damaging implications for the tokamak. Although
disruptions are difficult to model using physical simulations, machine learning
(ML) models have shown promise in predicting these phenomena. Here, we first
study several variations on masked autoregressive transformers, achieving an
average of 5\% increase in Area Under the Receiving Operating Characteristic
metric above existing methods. We then compare transformer models to limited
context neural networks in order to shed light on the ``memory'' of plasma
effected by tokamaks controls. With these model comparisons, we argue for the
persistence of a memory throughout the plasma \textit{in the context of
tokamaks} that our model exploits.
