Nitral-AI commited on
Commit
46acfd9
·
verified ·
1 Parent(s): 637ad36

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -5,4 +5,8 @@ language:
5
  base_model:
6
  - mistralai/Mistral-Small-3.2-24B-Instruct-2506
7
  ---
8
- Trained on https://huggingface.co/datasets/Nitral-AI/antirep_sharegpt for 2 epochs at 3e-5
 
 
 
 
 
5
  base_model:
6
  - mistralai/Mistral-Small-3.2-24B-Instruct-2506
7
  ---
8
+ # Trained on https://huggingface.co/datasets/Nitral-AI/antirep_sharegpt for 2 epochs at 3e-5, 4bit qlora. at rank/alpha 32.
9
+
10
+ ## Original set was created by: "Preference dataset meant to decrease repetition, measured as either copying n-grams from input or infinite / semi-infite repetition of tokens; the chosen split is V3 03/24 instructed to avoid n-gram repetition, while the rejected split consists of either V3 03/24 instructed to copy from the input or Qwen 3 8B with a rep pen of 0.7."
11
+
12
+ ## just took the preference rows, removed the rejected, converted to sharegpt jsonl to create the sft version of the set used above.