  We show how bidirectional transformers trained for masked token prediction
can be applied to neural image compression to achieve state-of-the-art results.
Such models were previously used for image generation by progressivly sampling
groups of masked tokens according to uncertainty-adaptive schedules. Unlike
these works, we demonstrate that predefined, deterministic schedules perform as
well or better for image compression. This insight allows us to use masked
attention during training in addition to masked inputs, and activation caching
during inference, to significantly speed up our models (~4 higher inference
speed) at a small increase in bitrate.
