  Large language models (LLMs) are now widely used in various fields, including
finance. However, Japanese financial-specific LLMs have not been proposed yet.
Hence, this study aims to construct a Japanese financial-specific LLM through
continual pre-training. Before tuning, we constructed Japanese
financial-focused datasets for continual pre-training. As a base model, we
employed a Japanese LLM that achieved state-of-the-art performance on Japanese
financial benchmarks among the 10-billion-class parameter models. After
continual pre-training using the datasets and the base model, the tuned model
performed better than the original model on the Japanese financial benchmarks.
Moreover, the outputs comparison results reveal that the tuned model's outputs
tend to be better than the original model's outputs in terms of the quality and
length of the answers. These findings indicate that domain-specific continual
pre-training is also effective for LLMs. The tuned model is publicly available
on Hugging Face.
