  Multi-modal program synthesis refers to the task of synthesizing programs
(code) from their specification given in different forms, such as a combination
of natural language and examples. Examples provide a precise but incomplete
specification, and natural language provides an ambiguous but more "complete"
task description. Machine-learned pre-trained models (PTMs) are adept at
handling ambiguous natural language, but struggle with generating syntactically
and semantically precise code. Program synthesis techniques can generate
correct code, often even from incomplete but precise specifications, such as
examples, but they are unable to work with the ambiguity of natural languages.
We present an approach that combines PTMs with component-based synthesis (CBS):
PTMs are used to generate candidates programs from the natural language
description of the task, which are then used to guide the CBS procedure to find
the program that matches the precise examples-based specification. We use our
combination approach to instantiate multi-modal synthesis systems for two
programming domains: the domain of regular expressions and the domain of CSS
selectors. Our evaluation demonstrates the effectiveness of our domain-agnostic
approach in comparison to a state-of-the-art specialized system, and the
generality of our approach in providing multi-modal program synthesis from
natural language and examples in different programming domains.
