  The availability of Large Language Models (LLMs) which can generate code, has
made it possible to create tools that improve developer productivity.
Integrated development environments or IDEs which developers use to write
software are often used as an interface to interact with LLMs. Although many
such tools have been released, almost all of them focus on general-purpose
programming languages. Domain-specific languages, such as those crucial for IT
automation, have not received much attention. Ansible is one such YAML-based IT
automation-specific language. Red Hat Ansible Lightspeed with IBM Watson Code
Assistant, further referred to as Ansible Lightspeed, is an LLM-based service
designed explicitly for natural language to Ansible code generation.
  In this paper, we describe the design and implementation of the Ansible
Lightspeed service and analyze feedback from thousands of real users. We
examine diverse performance indicators, classified according to both immediate
and extended utilization patterns along with user sentiments. The analysis
shows that the user acceptance rate of Ansible Lightspeed suggestions is higher
than comparable tools that are more general and not specific to a programming
language. This remains true even after we use much more stringent criteria for
what is considered an accepted model suggestion, discarding suggestions which
were heavily edited after being accepted. The relatively high acceptance rate
results in higher-than-expected user retention and generally positive user
feedback. This paper provides insights on how a comparatively small, dedicated
model performs on a domain-specific language and more importantly, how it is
received by users.
