  Simultaneous machine translation aims at solving the task of real-time
translation by starting to translate before consuming the full input, which
poses challenges in terms of balancing quality and latency of the translation.
The wait-$k$ policy offers a solution by starting to translate after consuming
$k$ words, where the choice of the number $k$ directly affects the latency and
quality. In applications where we seek to keep the choice over latency and
quality at inference, the wait-$k$ policy obliges us to train more than one
model. In this paper, we address the challenge of building one model that can
fulfil multiple latency levels and we achieve this by introducing lightweight
adapter modules into the decoder. The adapters are trained to be specialized
for different wait-$k$ values and compared to other techniques they offer more
flexibility to allow for reaping the benefits of parameter sharing and
minimizing interference. Additionally, we show that by combining with an
adaptive strategy, we can further improve the results. Experiments on two
language directions show that our method outperforms or competes with other
strong baselines on most latency values.
