cortecs/Qwen3-8B-NVFP4A16

Sparse FP4 Collection

This model is part of Cortecs' experimental model collection based on Qwen3.
This collection features 2:4 structured sparsity and NVFP4 / NVFP4-A16 quantization, optionally followed by light fine-tuning. The goal is to explore the trade-offs between compression, accuracy and throughput on Blackwell-class GPUs.

Model Description

The models are derived from the Qwen3 family and compressed using:

• 2:4 sparsity (50 percent structured sparsity)
• NVFP4 or NVFP4-A16 quantization
• Optional short fine-tuning to recover accuracy

These models target extremely high throughput on modern hardware while retaining useful accuracy for English and multilingual tasks.

Evaluation

All results were produced with a unified evaluation pipeline using standard academic benchmarks.

Benchmark Results

Model	ARC	Hellaswag	MMLU	ARC_de	Hellaswag_de	MMLU_de	TruthfulQA	CrowS	English Avg	German Avg	Safety Avg
Qwen3 8B	66.7	67.2	78.22	54.8	54.9	67.8	54.42	37.69	70.71	59.17	46.06
Qwen3 4B	63.3	62.5	73.07	47.5	49.9	65.1	54.76	41.03	66.29	54.17	47.90
Qwen3 8B NVFP4A16	66.4	66.5	75.54	54.2	54.4	67.7	53.72	38.04	69.48	58.77	45.88
Qwen3 8B NVFP4	66.3	66.6	75.54	54.4	54.3	68.1	53.76	37.92	69.48	58.93	45.84
Qwen3 8B Sparse NVFP4A16	50.5	57.4	53.35	30.7	36.0	34.4	46.95	39.89	53.75	33.70	43.42
Qwen3 8B Sparse Finetune 0.01	53.8	62.8	60.17	35.8	46.6	46.4	50.66	39.18	58.92	42.93	44.92
Qwen3 8B Sparse Finetune 0.1	56.4	62.2	60.89	38.9	46.2	44.0	52.13	38.04	59.83	43.03	45.09

Performance

Throughput measurements were conducted on a single B200 GPU.

Model	Total tokens/s
Qwen3 8B	30379
Qwen3 4B	34483
Qwen3 8B NVFP4A16	15978
Qwen3 8B Sparse NVFP4A16	15860
Qwen3 8B NVFP4	35296

Notes

• 2:4 structured sparsity always results in 50 percent zeroed weights.
• FP4 execution on Blackwell requires specialized kernels; throughput varies depending on backend support.
• Sparse FP4 models show reduced accuracy but improved efficiency. Light fine-tuning is essential to recover performance.

Intended Use

These models are experimental, designed only to evaluate sparsity and quantization strategies. They should not be used for production systems, safety-critical applications, or deployment scenarios involving real user data.

Limitations

Sparse FP4 models may exhibit reduced robustness and generalization.