  Characters do not convey meaning, but sequences of characters do. We propose
an unsupervised distributional method to learn the abstract meaningful units in
a sequence of characters. Rather than segmenting the sequence, our Dynamic
Capacity Slot Attention model discovers continuous representations of the
objects in the sequence, extending an architecture for object discovery in
images. We train our model on different languages and evaluate the quality of
the obtained representations with forward and reverse probing classifiers.
These experiments show that our model succeeds in discovering units which are
similar to those proposed previously in form, content and level of abstraction,
and which show promise for capturing meaningful information at a higher level
of abstraction.
