Dolphin-v2-f32-GGUF

ByteDance Dolphin-v2 is a 3B-parameter vision-language model built on Qwen2.5-VL-3B with Native Resolution Vision Transformer (NaViT) encoder and autoregressive decoder, designed as a universal document parsing solution via a document-type-aware two-stage architecture that classifies digital-born vs. photographed documents before applying hybrid strategies—element-wise parallel parsing for clean PDFs and holistic parsing for distorted scans. It supports 21 element categories (headings sec_0-5, paragraphs, formulas in LaTeX, HTML tables, indented code blocks, figures, lists, etc.) with absolute pixel coordinates for precise localization, achieving state-of-the-art OmniDocBench v1.5 scores of 89.45 overall (+14.78 over original Dolphin), 0.054 edit distance for text/reading order, 86.72% CDM for formulas, and 87.02/90.48 TEDS/TEDS-S for tables at 0.1729 FPS on 8-12GB VRAM GPUs. Specialized modules (P_formula, P_code, P_table, P_paragraph) enable structured JSON/Markdown/HTML outputs for privacy-focused local inference in healthcare/legal/finance, outperforming general VLMs in speed (2x faster) and accuracy across distortions, skews, and perspectives.

Dolphin-v2 [GGUF]

File Name	Quant Type	File Size	File Link
Dolphin-v2.BF16.gguf	BF16	6.18 GB	Download
Dolphin-v2.F32.gguf	F32	12.3 GB	Download
Dolphin-v2.IQ4_XS.gguf	IQ4_XS	1.75 GB	Download
Dolphin-v2.Q2_K.gguf	Q2_K	1.27 GB	Download
Dolphin-v2.Q3_K_L.gguf	Q3_K_L	1.71 GB	Download
Dolphin-v2.Q3_K_M.gguf	Q3_K_M	1.59 GB	Download
Dolphin-v2.Q3_K_S.gguf	Q3_K_S	1.45 GB	Download
Dolphin-v2.Q4_K_M.gguf	Q4_K_M	1.93 GB	Download
Dolphin-v2.Q4_K_S.gguf	Q4_K_S	1.83 GB	Download
Dolphin-v2.Q5_K_M.gguf	Q5_K_M	2.22 GB	Download
Dolphin-v2.Q5_K_S.gguf	Q5_K_S	2.17 GB	Download
Dolphin-v2.Q6_K.gguf	Q6_K	2.54 GB	Download
Dolphin-v2.Q8_0.gguf	Q8_0	3.29 GB	Download
Dolphin-v2.f16.gguf	F16	6.18 GB	Download
Dolphin-v2.i1-IQ1_M.gguf	i1-IQ1_M	850 MB	Download
Dolphin-v2.i1-IQ1_S.gguf	i1-IQ1_S	791 MB	Download
Dolphin-v2.i1-IQ2_M.gguf	i1-IQ2_M	1.14 GB	Download
Dolphin-v2.i1-IQ2_S.gguf	i1-IQ2_S	1.06 GB	Download
Dolphin-v2.i1-IQ2_XS.gguf	i1-IQ2_XS	1.03 GB	Download
Dolphin-v2.i1-IQ2_XXS.gguf	i1-IQ2_XXS	948 MB	Download
Dolphin-v2.i1-IQ3_M.gguf	i1-IQ3_M	1.49 GB	Download
Dolphin-v2.i1-IQ3_S.gguf	i1-IQ3_S	1.46 GB	Download
Dolphin-v2.i1-IQ3_XS.gguf	i1-IQ3_XS	1.39 GB	Download
Dolphin-v2.i1-IQ3_XXS.gguf	i1-IQ3_XXS	1.28 GB	Download
Dolphin-v2.i1-IQ4_NL.gguf	i1-IQ4_NL	1.83 GB	Download
Dolphin-v2.i1-IQ4_XS.gguf	i1-IQ4_XS	1.74 GB	Download
Dolphin-v2.i1-Q2_K.gguf	i1-Q2_K	1.27 GB	Download
Dolphin-v2.i1-Q2_K_S.gguf	i1-Q2_K_S	1.2 GB	Download
Dolphin-v2.i1-Q3_K_L.gguf	i1-Q3_K_L	1.71 GB	Download
Dolphin-v2.i1-Q3_K_M.gguf	i1-Q3_K_M	1.59 GB	Download
Dolphin-v2.i1-Q3_K_S.gguf	i1-Q3_K_S	1.45 GB	Download
Dolphin-v2.i1-Q4_0.gguf	i1-Q4_0	1.83 GB	Download
Dolphin-v2.i1-Q4_1.gguf	i1-Q4_1	2 GB	Download
Dolphin-v2.i1-Q4_K_M.gguf	i1-Q4_K_M	1.93 GB	Download
Dolphin-v2.i1-Q4_K_S.gguf	i1-Q4_K_S	1.83 GB	Download
Dolphin-v2.i1-Q5_K_M.gguf	i1-Q5_K_M	2.22 GB	Download
Dolphin-v2.i1-Q5_K_S.gguf	i1-Q5_K_S	2.17 GB	Download
Dolphin-v2.i1-Q6_K.gguf	i1-Q6_K	2.54 GB	Download
Dolphin-v2.imatrix.gguf	imatrix	3.39 MB	Download
Dolphin-v2.mmproj-Q8_0.gguf	mmproj-Q8_0	848 MB	Download
Dolphin-v2.mmproj-bf16.gguf	mmproj-bf16	1.34 GB	Download
Dolphin-v2.mmproj-f16.gguf	mmproj-f16	1.34 GB	Download
Dolphin-v2.mmproj-f32.gguf	mmproj-f32	2.67 GB	Download

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):