  Accuracy and efficiency remain challenges for multi-party computation (MPC)
frameworks. Spin is a GPU-accelerated MPC framework that supports multiple
computation parties and a dishonest majority adversarial setup. We propose
optimized protocols for non-linear functions that are critical for machine
learning, as well as several novel optimizations specific to attention that is
the fundamental unit of Transformer models, allowing Spin to perform
non-trivial CNNs training and Transformer inference without sacrificing
security. At the backend level, Spin leverages GPU, CPU, and RDMA-enabled smart
network cards for acceleration. Comprehensive evaluations demonstrate that Spin
can be up to $2\times$ faster than the state-of-the-art for deep neural network
training. For inference on a Transformer model with 18.9 million parameters,
our attention-specific optimizations enable Spin to achieve better efficiency,
less communication, and better accuracy.
