  Recently, high-performing code generation systems based on large language
models have surfaced. They are trained on massive corpora containing much more
natural text than actual executable computer code. This work shows that current
code generation systems exhibit undesired biases inherited from their large
language model backbones, which can reduce the quality of the generated code
under specific circumstances.
  To investigate the effect, we propose the "block of influence" concept, which
enables a modular decomposition and analysis of the coding challenges. We
introduce an automated intervention mechanism reminiscent of adversarial
testing that exposes undesired biases through the failure modes of the models
under test. Finally, we demonstrate how our framework can be used as a data
transformation technique during fine-tuning, acting as a mitigation strategy
for these biases.
