---
library_name: transformers
tags:
- compression
- expert-merging
- moe
- code
license: apache-2.0
base_model:
- Qwen/Qwen3-Coder-Next
---

# Qwen3-Coder-Next-REAP

This model is a compressed version of [Qwen/Qwen3-Coder-Next](https://huggingface.co/Qwen/Qwen3-Coder-Next). 
It is obtained by reducing the number of experts in each MoE layer from 512 to 384 using the REAP baseline method as described in https://bknyaz.github.io/blog/2026/moe/.

**Compared to other models obtained in this collection, more coding data is used in the calibration data during pruning/merging 
to better preserve original's model coding abilities. Specifically, the ratio between c4, math and coding data (see https://bknyaz.github.io/blog/2026/moe/) is 0.0, 0.7, 0.3.
The calibration data used here is the same as in our [Qwen3-Coder-Next-REAM](https://huggingface.co/SamsungSAILMontreal/Qwen3-Coder-Next-REAM).**

The compressed model has 60B params (120GB) instead of 80B (160GB) of the original model, 
reducing storage and GPU memory requirements by roughly 25%. At the same time, 
the model retains >=96% of the original model's performance on a variety of benchmarks (see Results section below).
Additional efficiency optimization (e.g., quantization) can be added similarly to the original model.

See additional details at [Qwen3-30B-A3B-Instruct-2507-REAM](https://huggingface.co/SamsungSAILMontreal/Qwen3-30B-A3B-Instruct-2507-REAM).

### Results

| Model                    | IFeval | AIME25 | GSM8K | GPQA-D | HumanEval | LiveCodeBench | AVG   |
|--------------------------|--------|--------|-------|--------|-----------|---------------|-------|
| Qwen3-Coder-Next         | 89.6   |   80.0 |  85.4 | 42.4   | 92.7      | 47.5          | 72.9  | 
| Qwen3-Coder-Next-REAP (this repo)    | 87.6   |   70.0 |  86.8 | 35.9   | 94.5      | 48.2          | 70.5  |
| [Qwen3-Coder-Next-REAM](https://huggingface.co/SamsungSAILMontreal/Qwen3-Coder-Next-REAM) | 89.3   |   80.0 |  85.3 | 40.4   | 94.5      | 48.0          | 72.9  |

## License

Please refer to the license of the original model [Qwen/Qwen3-Coder-Next](https://huggingface.co/Qwen/Qwen3-Coder-Next).