Document int4/int8 quants

README.md

@@ -36,3 +36,16 @@ The model spells numbers (هشت not ۸). Apply `persian_itn.py` at display for
 - [Koochik 114M](https://huggingface.co/Reza2kn/Shenava-Koochik-v1.0-tract-streaming) — flagship
 - [Rizeh 32M](https://huggingface.co/Reza2kn/Shenava-Rizeh-v1.0-tract-streaming) — mid
 - [Rizeh-Pizeh 6.9M](https://huggingface.co/Reza2kn/Shenava-Rizeh-Pizeh-v1.0-tract-streaming) — tiniest
+
+
+## Quantized variants — int4 / int8 (NEW)
+
+Our streaming support is **merged into tract main** ([sonos/tract#2441](https://github.com/sonos/tract/pull/2441)), which also ships int4 (`MatMulNBits` -> Q4_0) and int8 GEMM kernels. So tract main runs quantized versions of this streaming model:
+
+| file | precision | size | notes |
+|---|---|---|---|
+| `model.onnx` | fp32 | 116MB | reference |
+| `model.int4.onnx` | **int4** (MatMulNBits / Q4_0, weight-only) | **40MB** | ⭐ recommended — 2.9x smaller, ~fp32 speed, **byte-identical** decode |
+| `model.int8.onnx` | int8 (matmul-only, MatMulInteger) | 49MB | byte-identical; slower on small-batch streaming (per-matmul `DynamicQuantizeLinear`) — best for large-batch / offline, or CPUs where it wins |
+
+Both quants decode **byte-identically** to fp32. For edge/on-device streaming, use **`model.int4.onnx`** (weight-only, no per-matmul activation quant). Needs **tract main** — the streaming fixes are upstream now, so the bundled `.patch` is no longer required.