  Recently, program synthesis driven by large language models (LLMs) has become
increasingly popular. However, program synthesis for machine learning (ML)
tasks still poses significant challenges. This paper explores a novel form of
program synthesis, targeting ML programs, by combining LLMs and automated
machine learning (autoML). Specifically, our goal is to fully automate the
generation and optimization of the code of the entire ML workflow, from data
preparation to modeling and post-processing, utilizing only textual
descriptions of the ML tasks. To manage the length and diversity of ML
programs, we propose to break each ML program into smaller, manageable parts.
Each part is generated separately by the LLM, with careful consideration of
their compatibilities. To ensure compatibilities, we design a testing technique
for ML programs. Unlike traditional program synthesis, which typically relies
on binary evaluations (i.e., correct or incorrect), evaluating ML programs
necessitates more than just binary judgments. Therefore, we further assess ML
programs numerically and select the optimal programs from a range of candidates
using AutoML methods. In experiments across various ML tasks, our method
outperforms existing methods in 10 out of 12 tasks for generating ML programs.
In addition, autoML significantly improves the performance of the generated ML
programs. In experiments, given the textual task description, our method,
Text-to-ML, generates the complete and optimized ML program in a fully
autonomous process.
