---
license: other
license_name: flux-1-dev-non-commercial-license
license_link: LICENSE.md
base_model:
- black-forest-labs/FLUX.1-Kontext-dev
base_model_relation: quantized
library_name: diffusers
tags:
- sdnq
- flux
- 4-bit
---
4 bit (UINT4 with SVD rank 32) quantization of [black-forest-labs/FLUX.1-Kontext-dev](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev) using [SDNQ](https://github.com/vladmandic/sdnext/wiki/SDNQ-Quantization).  

Usage:  
```
pip install git+https://github.com/Disty0/sdnq
```

```py
import torch
import diffusers
from diffusers.utils import load_image
from sdnq import SDNQConfig # import sdnq to register it into diffusers and transformers

pipe = diffusers.FluxKontextPipeline.from_pretrained("Disty0/FLUX.1-Kontext-dev-SDNQ-uint4-svd-r32", torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()

input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png")
image = pipe(
    image=input_image,
    prompt="Add a hat to the cat",
    guidance_scale=2.5,
    generator=torch.manual_seed(0),
).images[0]
image.save("flux-kontext-dev-sdnq-uint4-svd-r32.png.png")
```

Original BF16 vs SDNQ quantization comparison:  

| Quantization | Model Size | Visualization |
| --- | --- | --- |
| Input Image | - | ![Input Image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png) |
| Original BF16 | 23.8 GB | ![Original BF16](https://cdn-uploads.huggingface.co/production/uploads/6456af6195082f722d178522/m7TvvvswC2lOsNV1a-CSv.png) |
| SDNQ UINT4 | 6.8 GB | ![SDNQ UINT4](https://cdn-uploads.huggingface.co/production/uploads/6456af6195082f722d178522/OXWJhg7FV5RmM5-PTi3DD.png) |