Trained on https://huggingface.co/datasets/Nitral-AI/antirep_sharegpt for 2 epochs at 3e-5, 4bit qlora. at rank/alpha 32.
Original set was created by: "Preference dataset meant to decrease repetition, measured as either copying n-grams from input or infinite / semi-infite repetition of tokens; the chosen split is V3 03/24 instructed to avoid n-gram repetition, while the rejected split consists of either V3 03/24 instructed to copy from the input or Qwen 3 8B with a rep pen of 0.7."
just took the preference rows, removed the rejected, converted to sharegpt jsonl to create the sft version of the set used above.
- Downloads last month
- -
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
๐
Ask for provider support
Model tree for Nitral-Private/Mistral-3.2-Instruct-Antirep-sft-24B-0.1
Base model
mistralai/Mistral-Small-3.1-24B-Base-2503