  Generative Large Language Models (LLMs) show potential in data analysis, yet
their full capabilities remain uncharted. Our work explores the capabilities of
LLMs for creating and refining visualizations via conversational interfaces. We
used an LLM to conduct a re-analysis of a prior Wizard-of-Oz study examining
the use of chatbots for conducting visual analysis. We surfaced the strengths
and weaknesses of LLM-driven analytic chatbots, finding that they fell short in
supporting progressive visualization refinements. From these findings, we
developed AI Threads, a multi-threaded analytic chatbot that enables analysts
to proactively manage conversational context and improve the efficacy of its
outputs. We evaluate its usability through a crowdsourced study (n=40) and
in-depth interviews with expert analysts (n=10). We further demonstrate the
capabilities of AI Threads on a dataset outside the LLM's training corpus. Our
findings show the potential of LLMs while also surfacing challenges and
fruitful avenues for future research.
