  Neural code generation systems have recently attracted increasing attention
to improve developer productivity and speed up software development. Typically,
these systems maintain a pre-trained neural model and make it available to
general users as a service (e.g., through remote APIs) and incorporate a
feedback mechanism to extensively collect and utilize the users' reaction to
the generated code, i.e., user feedback. However, the security implications of
such feedback have not yet been explored. With a systematic study of current
feedback mechanisms, we find that feedback makes these systems vulnerable to
feedback data injection (FDI) attacks. We discuss the methodology of FDI
attacks and present a pre-attack profiling strategy to infer the attack
constraints of a targeted system in the black-box setting. We demonstrate two
proof-of-concept examples utilizing the FDI attack surface to implement prompt
injection attacks and backdoor attacks on practical neural code generation
systems. The attacker may stealthily manipulate a neural code generation system
to generate code with vulnerabilities, attack payload, and malicious and spam
messages. Our findings reveal the security implications of feedback mechanisms
in neural code generation systems, paving the way for increasing their
security.
