| license: apache-2.0 | |
| base_model: Qwen/Qwen2.5-Coder-7B-Instruct | |
| tags: | |
| - gguf | |
| - quantized | |
| - q4_k_m | |
| # Qwen2.5-Coder-7B-Instruct-iat-05-1-GGUF | |
| This is a GGUF quantized version (q4_k_m) of Qwen/Qwen2.5-Coder-7B-Instruct fine-tuned with the 'iat-05-1' adapter. | |
| ## Model Details | |
| - **Base Model:** Qwen/Qwen2.5-Coder-7B-Instruct | |
| - **Adapter:** iat-05-1 | |
| - **Quantization:** q4_k_m | |
| - **Format:** GGUF | |
| ## Usage | |
| This model can be used with llama.cpp or any compatible inference engine that supports GGUF format. | |
| ```bash | |
| # Example with llama.cpp | |
| ./llama-cli -m Qwen2.5-Coder-7B-Instruct-iat-05-1-q4_k_m.gguf -p "Your prompt here" | |
| ``` | |
| ## Files | |
| - `Qwen2.5-Coder-7B-Instruct-iat-05-1-q4_k_m.gguf` - Quantized model in GGUF format (q4_k_m) | |