  The fields of both Natural Language Processing (NLP) and Automated Machine
Learning (AutoML) have achieved remarkable results over the past years. In NLP,
especially Large Language Models (LLMs) have experienced a rapid series of
breakthroughs very recently. We envision that the two fields can radically push
the boundaries of each other through tight integration. To showcase this
vision, we explore the potential of a symbiotic relationship between AutoML and
LLMs, shedding light on how they can benefit each other. In particular, we
investigate both the opportunities to enhance AutoML approaches with LLMs from
different perspectives and the challenges of leveraging AutoML to further
improve LLMs. To this end, we survey existing work, and we critically assess
risks. We strongly believe that the integration of the two fields has the
potential to disrupt both fields, NLP and AutoML. By highlighting conceivable
synergies, but also risks, we aim to foster further exploration at the
intersection of AutoML and LLMs.
