  Roof-mounted spinning LiDAR sensors are widely used by autonomous vehicles.
However, most semantic datasets and algorithms used for LiDAR sequence
segmentation operate on $360^\circ$ frames, causing an acquisition latency
incompatible with real-time applications. To address this issue, we first
introduce HelixNet, a $10$ billion point dataset with fine-grained labels,
timestamps, and sensor rotation information necessary to accurately assess the
real-time readiness of segmentation algorithms. Second, we propose Helix4D, a
compact and efficient spatio-temporal transformer architecture specifically
designed for rotating LiDAR sequences. Helix4D operates on acquisition slices
corresponding to a fraction of a full sensor rotation, significantly reducing
the total latency. Helix4D reaches accuracy on par with the best segmentation
algorithms on HelixNet and SemanticKITTI with a reduction of over $5\times$ in
terms of latency and $50\times$ in model size. The code and data are available
at: https://romainloiseau.fr/helixnet
