  Large Language Models (LLMs) have shown remarkable capabilities across a wide
variety of Natural Language Processing (NLP) tasks and have attracted attention
from multiple domains, including financial services. Despite the extensive
research into general-domain LLMs, and their immense potential in finance,
Financial LLM (FinLLM) research remains limited. This survey provides a
comprehensive overview of FinLLMs, including their history, techniques,
performance, and opportunities and challenges. Firstly, we present a
chronological overview of general-domain Pre-trained Language Models (PLMs)
through to current FinLLMs, including the GPT-series, selected open-source
LLMs, and financial LMs. Secondly, we compare five techniques used across
financial PLMs and FinLLMs, including training methods, training data, and
fine-tuning methods. Thirdly, we summarize the performance evaluations of six
benchmark tasks and datasets. In addition, we provide eight advanced financial
NLP tasks and datasets for developing more sophisticated FinLLMs. Finally, we
discuss the opportunities and the challenges facing FinLLMs, such as
hallucination, privacy, and efficiency. To support AI research in finance, we
compile a collection of accessible datasets and evaluation benchmarks on
GitHub.
