  This paper presents a framework which enables a user to more easily make
corrections to adversarial texts. While attack algorithms have been
demonstrated to automatically build adversaries, changes made by the algorithms
can often have poor semantics or syntax. Our framework is designed to
facilitate human intervention by aiding users in making corrections. The
framework extends existing attack algorithms to work within an evolutionary
attack process paired with a visual analytics loop. Using an interactive
dashboard a user is able to review the generation process in real time and
receive suggestions from the system for edits to be made. The adversaries can
be used to both diagnose robustness issues within a single classifier or to
compare various classifier options. With the weaknesses identified, the
framework can also be used as a first step in mitigating adversarial threats.
The framework can be used as part of further research into defense methods in
which the adversarial examples are used to evaluate new countermeasures. We
demonstrate the framework with a word swapping attack for the task of sentiment
classification.
