exllamav2 quantizations of ArliAI's DS-R1-Distill-70B-ArliAI-RpR-v4-Large
2.25bpw h6 (20.678 GiB)
4.25bpw h6 (36.616 GiB)
6.00bpw h6 (50.561 GiB)
8.00bpw h8 (66.658 GiB) (Queued for upload)
measurement.json
Model tree for MikeRoz/DS-R1-Distill-70B-ArliAI-RpR-v4-Large-exl2
Base model
deepseek-ai/DeepSeek-R1-Distill-Llama-70B