Update README.md

README.md

@@ -1,3 +1,17 @@
----
-license: apache-2.0
----
+
+---
+license: apache-2.0
+---
+
+This repository contains 3 versions of Qwen3-30B-A3B quantized with `IQ4_KS`. The interesting part is that these models achieve a lower peplexity on
+`wiki.test.raw` than the original `bf16` model. This is interesting, considering that no QAT has been mentioned
+in the [Qwen3 announcement](https://qwenlm.github.io/blog/qwen3/). Hence I'm putting them out there for anyone interested in evaluating performance by means other than PPL,
+or just using for local inferrence.
+For more details see [this discussion](https://github.com/ikawrakow/ik_llama.cpp/discussions/359).
+
+**Note**: These models will only work with [ik_llama.cpp](https://github.com/ikawrakow/ik_llama.cpp) as the `IQ4_KS` quantization type is not available in mainline `llama.cpp`.
+
+The only difference between the 3 models is the imatrix used:
+* **Qwen3-30B-A3B-IQ4_KS-IK-Imatrix.gguf**: imatrix computed using 500,000 tokens from `wiki.train.raw`
+* **Qwen3-30B-A3B-IQ4_KS-Bartowski-Imatrix.gguf**: Bartowski's [imatrix](https://huggingface.co/bartowski/Qwen_Qwen3-30B-A3B-GGUF/blob/main/Qwen_Qwen3-30B-A3B.imatrix)
+* **Qwen3-30B-A3B-IQ4_KS-Unsloth-Imatrix.gguf**: Unsloth [imatrix](https://huggingface.co/unsloth/Qwen3-30B-A3B-128K-GGUF/blob/main/imatrix_unsloth.dat)