  Behavior-driven development (BDD) is an Agile testing methodology fostering
collaboration among developers, QA analysts, and stakeholders. In this
manuscript, we propose a novel approach to enhance BDD practices using large
language models (LLMs) to automate acceptance test generation. Our study uses
zero and few-shot prompts to evaluate LLMs such as GPT-3.5, GPT-4, Llama-2-13B,
and PaLM-2. The paper presents a detailed methodology that includes the
dataset, prompt techniques, LLMs, and the evaluation process. The results
demonstrate that GPT-3.5 and GPT-4 generate error-free BDD acceptance tests
with better performance. The few-shot prompt technique highlights its ability
to provide higher accuracy by incorporating examples for in-context learning.
Furthermore, the study examines syntax errors, validation accuracy, and
comparative analysis of LLMs, revealing their effectiveness in enhancing BDD
practices. However, our study acknowledges that there are limitations to the
proposed approach. We emphasize that this approach can support collaborative
BDD processes and create opportunities for future research into automated BDD
acceptance test generation using LLMs.
