  Large Language Models (LLMs) have shown great success in code generation.
LLMs take as the input a prompt and output the code. A key question is how to
make prompts (i.e., Prompting Techniques). Existing prompting techniques are
designed for natural language generation and have low accuracy in code
generation.
  In this paper, we propose a new prompting technique named AceCoder. Our
motivation is that code generation meets two unique challenges (i.e.,
requirement understanding and code implementation). AceCoder contains two novel
mechanisms (i.e., guided code generation and example retrieval) to solve these
challenges. (1) Guided code generation asks LLMs first to analyze requirements
and output an intermediate preliminary (e.g., test cases). The preliminary is
used to clarify requirements and tell LLMs "what to write". (2) Example
retrieval selects similar programs as examples in prompts, which provide lots
of relevant content (e.g., algorithms, APIs) and teach LLMs "how to write". We
apply AceCoder to three LLMs (e.g., Codex) and evaluate it on three public
benchmarks using the Pass@k. Results show that AceCoder can significantly
improve the performance of LLMs on code generation. (1) In terms of Pass@1,
AceCoder outperforms the state-of-the-art baseline by up to 56.4% in MBPP,
70.7% in MBJP, and 88.4% in MBJSP. (2) AceCoder is effective in LLMs with
different sizes (i.e., 6B to 13B) and different languages (i.e., Python, Java,
and JavaScript). (3) Human evaluation shows human developers prefer programs
from AceCoder.
