|
|
--- |
|
|
license: other |
|
|
base_model: Qwen/Qwen3-4B-Instruct-2507 |
|
|
datasets: |
|
|
- custom |
|
|
library_name: transformers |
|
|
tags: |
|
|
- sft |
|
|
- instruction-tuning |
|
|
- qwen |
|
|
inference: false |
|
|
--- |
|
|
|
|
|
# qwen3_4b_instruct_2507_sft_v1 |
|
|
|
|
|
This repository contains the supervised fine-tuning (SFT) checkpoint for a Qwen3 4B Instruct model trained with DeepSpeed ZeRO-3. The weights have been consolidated and exported to the Hugging Face safetensors format for easier deployment. |
|
|
|
|
|
## Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
repo_id = "Chouoftears/qwen3_4b_instruct_2507_sft_v1" |
|
|
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True) |
|
|
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True) |
|
|
``` |
|
|
|
|
|
## Training |
|
|
|
|
|
- Base model: `Qwen/Qwen3-4B-Instruct-2507` |
|
|
- Framework: `transformers==4.56.2` |
|
|
- Optimization: DeepSpeed ZeRO Stage-3, bf16 |
|
|
- SFT run name: `qwen3-4B-Instruct-2507-toucan-sft-3ep` |
|
|
- Max sequence length: 262,144 tokens (per config) |
|
|
|
|
|
Refer to `training_args.bin` in the original run directory for the full trainer configuration. |
|
|
|
|
|
## Files |
|
|
|
|
|
- `model-0000X-of-00004.safetensors`: model weights shards |
|
|
- `model.safetensors.index.json`: weight index map |
|
|
- `config.json` / `generation_config.json`: architecture and generation defaults |
|
|
- Tokenizer artifacts: `tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt`, `special_tokens_map.json`, `added_tokens.json` |
|
|
- `chat_template.jinja`: conversation formatting used during SFT |
|
|
|
|
|
## Limitations |
|
|
|
|
|
This checkpoint inherits limitations from the base Qwen3 model and SFT data. Review and align with your downstream safety and compliance requirements before deployment. |
|
|
|
