  Adversarial attacks have gained traction in order to identify potential
vulnerabilities in neural ranking models (NRMs), but current attack methods
often introduce grammatical errors, nonsensical expressions, or incoherent text
fragments, which can be easily detected. Additionally, current methods rely
heavily on the use of a well-imitated surrogate NRM to guarantee the attack
effect, which makes them difficult to use in practice. To address these issues,
we propose a framework called Imperceptible DocumEnt Manipulation (IDEM) to
produce adversarial documents that are less noticeable to both algorithms and
humans. IDEM instructs a well-established generative language model, such as
BART, to generate connection sentences without introducing easy-to-detect
errors, and employs a separate position-wise merging strategy to balance
relevance and coherence of the perturbed text. Experimental results on the
popular MS MARCO benchmark demonstrate that IDEM can outperform strong
baselines while preserving fluency and correctness of the target documents as
evidenced by automatic and human evaluations. Furthermore, the separation of
adversarial text generation from the surrogate NRM makes IDEM more robust and
less affected by the quality of the surrogate NRM.
