metadata
language:
- en
library_name: mlx
tags:
- qwen-coder
- MOE
- pruning
- compression
- mlx
license: apache-2.0
name: cerebras/Qwen3-Coder-REAP-25B-A3B
description: >
This model was obtained by uniformly pruning 20% of experts in
Qwen3-Coder-30B-A3B-Instruct using the REAP method.
readme: |
https://huggingface.co/cerebras/Qwen3-Coder-REAP-25B-A3B/main/README.md
license_link: https://huggingface.co/cerebras/Qwen3-Coder-REAP-25B-A3B/blob/main/LICENSE
pipeline_tag: text-generation
base_model: cerebras/Qwen3-Coder-REAP-25B-A3B
Qwen3-Coder-REAP-25B-A3B-qx65x-hi-mlx
This version of the Deckard(qx) formula uses embeddings at 6 bit, along with the head and select attention paths, leaving the rest at 5 bit.
The model is quantized with group size 32(hi).
It is aimed as a mid-range quant with a quality approaching q8, that would run comfortably on a smaller Mac.
This is an update from the model: Qwen3-Coder-REAP-25B-A3B-qx64-hi-mlx that uses the base and embeddings at 4 bit.
Metrics coming soon.
-G
This model Qwen3-Coder-REAP-25B-A3B-qx65x-hi-mlx was converted to MLX format from cerebras/Qwen3-Coder-REAP-25B-A3B using mlx-lm version 0.28.3.
Use with mlx
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("Qwen3-Coder-REAP-25B-A3B-qx65x-hi-mlx")
prompt = "hello"
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True
)
response = generate(model, tokenizer, prompt=prompt, verbose=True)