  Although it has been demonstrated that Natural Language Processing (NLP)
algorithms are vulnerable to deliberate attacks, the question of whether such
weaknesses can lead to software security threats is under-explored. To bridge
this gap, we conducted vulnerability tests on Text-to-SQL systems that are
commonly used to create natural language interfaces to databases. We showed
that the Text-to-SQL modules within six commercial applications can be
manipulated to produce malicious code, potentially leading to data breaches and
Denial of Service attacks. This is the first demonstration that NLP models can
be exploited as attack vectors in the wild. In addition, experiments using four
open-source language models verified that straightforward backdoor attacks on
Text-to-SQL systems achieve a 100% success rate without affecting their
performance. The aim of this work is to draw the community's attention to
potential software security issues associated with NLP algorithms and encourage
exploration of methods to mitigate against them.
