SmolLM2-Rethink-360M-GGUF

SmolLM2-Rethink-360M is an experimental lightweight reasoning model trained on the Celestia3-DeepSeek-R1-0528 dataset. Built on top of the SmolLM2-135M-Instruct architecture and scaled to 360M parameters, it is designed to enhance lightweight reasoning, logical deduction, and structured response generation—all while maintaining efficiency for resource-constrained environments.

Model Files

File Name	Size	Type	Description
SmolLM2-Rethink-360M.Q2_K.gguf	219 MB	Model	Q2_K quantized model (smallest)
SmolLM2-Rethink-360M.Q3_K_S.gguf	219 MB	Model	Q3_K_S quantized model
SmolLM2-Rethink-360M.Q3_K_M.gguf	235 MB	Model	Q3_K_M quantized model
SmolLM2-Rethink-360M.Q3_K_L.gguf	246 MB	Model	Q3_K_L quantized model
SmolLM2-Rethink-360M.Q4_K_S.gguf	260 MB	Model	Q4_K_S quantized model
SmolLM2-Rethink-360M.Q4_K_M.gguf	271 MB	Model	Q4_K_M quantized model
SmolLM2-Rethink-360M.Q5_K_S.gguf	283 MB	Model	Q5_K_S quantized model
SmolLM2-Rethink-360M.Q5_K_M.gguf	290 MB	Model	Q5_K_M quantized model
SmolLM2-Rethink-360M.Q6_K.gguf	367 MB	Model	Q6_K quantized model
SmolLM2-Rethink-360M.Q8_0.gguf	386 MB	Model	Q8_0 quantized model
SmolLM2-Rethink-360M.BF16.gguf	726 MB	Model	BF16 precision model
SmolLM2-Rethink-360M.F16.gguf	726 MB	Model	F16 precision model
SmolLM2-Rethink-360M.F32.gguf	1.45 GB	Model	F32 full precision model (largest)
.gitattributes	2.4 kB	Config	Git LFS configuration
config.json	29 Bytes	Config	Model configuration
README.md	31 Bytes	Documentation	Repository documentation

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):