  Research in the field of language models is rapidly evolving, with many open
models being released to the public. Openly available pretraining corpora
usually focus on only a handful of languages, with many others either missing
completely or extremely underrepresented. In this report, we introduce FuLG, a
hundred-fifty-billion-token Romanian corpus extracted from CommonCrawl. We
present our methodology for filtering FuLG and compare it via ablation studies
against existing Romanian corpora.
