Qwen3-MoE
Collection
Compressed Qwen3 MoE models with a reduced number of experts. See additional models at https://huggingface.co/bknyaz.
•
7 items
•
Updated
Qwen3-Next-80B-A3B-Instruct-REAP
This model is a compressed version of Qwen/Qwen3-Next-80B-A3B-Instruct. It is obtained by reducing the number of experts in each MoE layer from 512 to 384 using the REAP baseline method as described in https://bknyaz.github.io/blog/2026/moe/. The compressed model has 60B params (120GB) instead of 80B (160GB) of the original model, reducing storage and GPU memory requirements by roughly 25%. At the same time, the model retains >=94% of the original model's performance on a variety of benchmarks (see Results section below). Additional efficiency optimization (e.g., quantization) can be added similarly to the original model.
See additional details at Qwen3-30B-A3B-Instruct-2507-REAM.
Results
| Model | IFeval | AIME25 | GSM8K | GPQA-D | HumanEval | LiveCodeBench | AVG |
|---|---|---|---|---|---|---|---|
| Qwen3-Next-80B-A3B-Instruct | 93.4 | 80.0 | 78.6 | 47.0 | 95.1 | 43.2 | 72.9 |
| Qwen3-Next-80B-A3B-Instruct-REAP | 91.0 | 66.7 | 78.8 | 37.9 | 91.5 | 45.0 | 68.5 |
License
Please refer to the license of the original model Qwen/Qwen3-Next-80B-A3B-Instruct.
- Downloads last month
- 52