  We present a novel architecture, residual attention net (RAN), which merges a
sequence architecture, universal transformer, and a computer vision
architecture, residual net, with a high-way architecture for cross-domain
sequence modeling. The architecture aims at addressing the long dependency
issue often faced by recurrent-neural-net-based structures. This paper serves
as a proof-of-concept for a new architecture, with RAN aiming at providing the
model a higher level understanding of sequence patterns. To our best knowledge,
we are the first to propose such an architecture. Out of the standard 85 UCR
data sets, we have achieved 35 state-of-the-art results with 10 results
matching current state-of-the-art results without further model fine-tuning.
The results indicate that such architecture is promising in complex,
long-sequence modeling and may have vast, cross-domain applications.
