Update README.md
Browse files
README.md
CHANGED
|
@@ -5,4 +5,8 @@ language:
|
|
| 5 |
base_model:
|
| 6 |
- mistralai/Mistral-Small-3.2-24B-Instruct-2506
|
| 7 |
---
|
| 8 |
-
Trained on https://huggingface.co/datasets/Nitral-AI/antirep_sharegpt for 2 epochs at 3e-5
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
base_model:
|
| 6 |
- mistralai/Mistral-Small-3.2-24B-Instruct-2506
|
| 7 |
---
|
| 8 |
+
# Trained on https://huggingface.co/datasets/Nitral-AI/antirep_sharegpt for 2 epochs at 3e-5, 4bit qlora. at rank/alpha 32.
|
| 9 |
+
|
| 10 |
+
## Original set was created by: "Preference dataset meant to decrease repetition, measured as either copying n-grams from input or infinite / semi-infite repetition of tokens; the chosen split is V3 03/24 instructed to avoid n-gram repetition, while the rejected split consists of either V3 03/24 instructed to copy from the input or Qwen 3 8B with a rep pen of 0.7."
|
| 11 |
+
|
| 12 |
+
## just took the preference rows, removed the rejected, converted to sharegpt jsonl to create the sft version of the set used above.
|