  In this paper, we explore the following question: Are synonym substitution
attacks really synonym substitution attacks (SSAs)? We approach this question
by examining how SSAs replace words in the original sentence and show that
there are still unresolved obstacles that make current SSAs generate invalid
adversarial samples. We reveal that four widely used word substitution methods
generate a large fraction of invalid substitution words that are ungrammatical
or do not preserve the original sentence's semantics. Next, we show that the
semantic and grammatical constraints used in SSAs for detecting invalid word
replacements are highly insufficient in detecting invalid adversarial samples.
