  A fundamental issue in machine learning is the robustness of the model with
respect to changes in the input. In natural language processing, models
typically contain a first embedding layer, transforming a sequence of tokens
into vector representations. While the robustness with respect to changes of
continuous inputs is well-understood, the situation is less clear when
considering discrete changes, for instance replacing a word by another in an
input sentence. Our work formally proves that popular embedding schemes, such
as concatenation, TF-IDF, and Paragraph Vector (a.k.a. doc2vec), exhibit
robustness in the H\"older or Lipschitz sense with respect to the Hamming
distance. We provide quantitative bounds for these schemes and demonstrate how
the constants involved are affected by the length of the document. These
findings are exemplified through a series of numerical examples.
