  This paper proposes a novel framework for digital curation of Web corpora in
order to provide robust estimation of their parameters, such as their
composition and the lexicon. In recent years language models pre-trained on
large corpora emerged as clear winners in numerous NLP tasks, but no proper
analysis of the corpora which led to their success has been conducted. The
paper presents a procedure for robust frequency estimation, which helps in
establishing the core lexicon for a given corpus, as well as a procedure for
estimating the corpus composition via unsupervised topic models and via
supervised genre classification of Web pages. The results of the digital
curation study applied to several Web-derived corpora demonstrate their
considerable differences. First, this concerns different frequency bursts which
impact the core lexicon obtained from each corpus. Second, this concerns the
kinds of texts they contain. For example, OpenWebText contains considerably
more topical news and political argumentation in comparison to ukWac or
Wikipedia. The tools and the results of analysis have been released.
