  The Object Constraint Language (OCL) is a declarative language that adds
constraints and object query expressions to MOF models. Despite its potential
to provide precision and conciseness to UML models, the unfamiliar syntax of
OCL has hindered its adoption. Recent advancements in LLMs, such as GPT-3, have
shown their capability in many NLP tasks, including semantic parsing and text
generation. Codex, a GPT-3 descendant, has been fine-tuned on publicly
available code from GitHub and can generate code in many programming languages.
We investigate the reliability of OCL constraints generated by Codex from
natural language specifications. To achieve this, we compiled a dataset of 15
UML models and 168 specifications and crafted a prompt template with slots to
populate with UML information and the target task, using both zero- and
few-shot learning methods. By measuring the syntactic validity and execution
accuracy metrics of the generated OCL constraints, we found that enriching the
prompts with UML information and enabling few-shot learning increases the
reliability of the generated OCL constraints. Furthermore, the results reveal a
close similarity based on sentence embedding between the generated OCL
constraints and the human-written ones in the ground truth, implying a level of
clarity and understandability in the generated OCL constraints by Codex.
