  Token free approaches have been successfully applied to a series of word and
span level tasks. In this work, we compare a byte-level (ByT5) and a wordpiece
based (mT5) sequence to sequence model on the 51 languages of the MASSIVE
multilingual semantic parsing dataset. We examine multiple experimental
settings: (i) zero-shot, (ii) full gold data and (iii) zero-shot with synthetic
data. By leveraging a state-of-the-art label projection method for machine
translated examples, we are able to reduce the gap in exact match accuracy to
only 5 points with respect to a model trained on gold data from all the
languages. We additionally provide insights on the cross-lingual transfer of
ByT5 and show how the model compares with respect to mT5 across all parameter
sizes.
