  Automated code generation can be a powerful technique for software
development, significantly reducing developers' efforts and time required to
create new code by generating it automatically based on requirements. Recently,
OpenAI's language model ChatGPT has emerged as a powerful tool for generating
human-like responses to a wide range of textual inputs (i.e., prompts),
including those related to code generation. However, the effectiveness of
ChatGPT for code generation is not well understood, and the generation
performance could be heavily influenced by the choice of prompt. To answer
these questions, we conducted experiments using the CodeXGlue dataset to
evaluate ChatGPT's capabilities for two code generation tasks, including
text-to-code and code-to-code generation. We designed prompts by leveraging the
chain-of-thought strategy with multi-step optimizations. Our results showed
that by carefully designing prompts to guide ChatGPT, the generation
performance can be improved substantially. We also analyzed the factors that
influenced the prompt design and provided insights that could guide future
research.
