  Transformer models have recently become very successful in the natural
language domain. Their value as sequence-to-sequence translators there, also
makes them a highly interesting technique for learning relationships between
astrophysical time series. Our aim is investigate how well such a transformer
neural network can establish causal temporal relations between different
channels of a single-source signal. We thus apply a transformer model to the
two phases of Gamma-Ray Bursts (GRBs), reconstructing one phase from the other.
GRBs are unique instances where a single process and event produces two
distinct time variable phenomena: the prompt emission and the afterglow. We
here investigate if a transformer model can predict the afterglow flux from the
prompt emission. If successful, such a predictive scheme might then be
distilled to the most important underlying physics drivers in the future. We
combine the transformer model with a novel dense neural network setup to
directly estimate the starting value of the prediction. We find that the
transformer model can, in some instances, successfully predict different phases
of canonical afterglows, including the plateau phase. Hence it is a useful and
promising new astrophysical analysis technique. For the GRB test case, the
method marginally exceeds the baseline model overall, but still achieves
accurate recovery of the prompt-afterglow fluence-fluence correlation in
reconstructed light curves. Despite this progress, we conclude that consistent
improvement over the baseline model is not yet achieved for the GRB case. We
discuss the future improvements in data and modeling that are required to
identify new physical-relation parameters or new insights into the single
process driving both GRB phases.
