  Large Language Models (LLMs) are becoming crucial across various fields,
emphasizing the urgency for high-quality models in underrepresented languages.
This study explores the unique challenges faced by low-resource languages, such
as data scarcity, model selection, evaluation, and computational limitations,
with a special focus on Turkish. We conduct an in-depth analysis to evaluate
the impact of training strategies, model choices, and data availability on the
performance of LLMs designed for underrepresented languages. Our approach
includes two methodologies: (i) adapting existing LLMs originally pretrained in
English to understand Turkish, and (ii) developing a model from the ground up
using Turkish pretraining data, both supplemented with supervised fine-tuning
on a novel Turkish instruction-tuning dataset aimed at enhancing reasoning
capabilities. The relative performance of these methods is evaluated through
the creation of a new leaderboard for Turkish LLMs, featuring benchmarks that
assess different reasoning and knowledge skills. Furthermore, we conducted
experiments on data and model scaling, both during pretraining and fine-tuning,
simultaneously emphasizing the capacity for knowledge transfer across languages
and addressing the challenges of catastrophic forgetting encountered during
fine-tuning on a different language. Our goal is to offer a detailed guide for
advancing the LLM framework in low-resource linguistic contexts, thereby making
natural language processing (NLP) benefits more globally accessible.
