HuggingFaceTB
/

cosmo2-tokenizer

Model card Files Files and versions

loubnabnl HF Staff commited on Sep 2, 2024

Commit

4ce2318

·

verified ·

1 Parent(s): 6f9fdab

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -6,7 +6,7 @@ datasets:
 # cosmo2-tokenizer
  Tokenizer for the training of cosmo2. This tokenizer was trained on 1M samples from:
- - FineWeb-Edu 80%
  - Cosmopedia v2 15%
  - StarCoderData  8%
  - OpenWebMath 5%

 # cosmo2-tokenizer
  Tokenizer for the training of cosmo2. This tokenizer was trained on 1M samples from:
+ - FineWeb-Edu 70%
  - Cosmopedia v2 15%
  - StarCoderData  8%
  - OpenWebMath 5%