Hugging Face's logo

EricB
/
gemma-1.1-2b-it-UQFF

gemma
uqff
mistral.rs
Model card Files
xet
Community
EricB HF Staff commited on
Commit
c81f881
·
verified ·
1 Parent(s): 3cb8fe7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -19,12 +19,12 @@ Run with [mistral.rs](https://github.com/EricLBuehler/mistral.rs). Documentation
19
  3) **Customizable** 🛠️: Make and publish your own UQFF files in minutes.
20
  ## Files
21
 
22
- |Name|Quantization type(s)|Example|
23
- |--|--|--|
24
- |gemma1.1-2b-instruct-f8e4m3.uqff|FP8|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff EricB/gemma-1.1-2b-it-UQFF/gemma1.1-2b-instruct-f8e4m3.uqff`|
25
- |gemma1.1-2b-instruct-hqq4.uqff|HQQ4|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff EricB/gemma-1.1-2b-it-UQFF/gemma1.1-2b-instruct-hqq4.uqff`|
26
- |gemma1.1-2b-instruct-hqq8.uqff|HQQ8|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff EricB/gemma-1.1-2b-it-UQFF/gemma1.1-2b-instruct-hqq8.uqff`|
27
- |gemma1.1-2b-instruct-q3k.uqff|Q3K|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff EricB/gemma-1.1-2b-it-UQFF/gemma1.1-2b-instruct-q3k.uqff`|
28
- |gemma1.1-2b-instruct-q4k.uqff|Q4K|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff EricB/gemma-1.1-2b-it-UQFF/gemma1.1-2b-instruct-q4k.uqff`|
29
- |gemma1.1-2b-instruct-q5k.uqff|Q5K|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff EricB/gemma-1.1-2b-it-UQFF/gemma1.1-2b-instruct-q5k.uqff`|
30
- |gemma1.1-2b-instruct-q8_0.uqff|Q8_0|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff EricB/gemma-1.1-2b-it-UQFF/gemma1.1-2b-instruct-q8_0.uqff`|
 
19
  3) **Customizable** 🛠️: Make and publish your own UQFF files in minutes.
20
  ## Files
21
 
22
+ |Quantization type(s)|Example|
23
+ |--|--|
24
+ |FP8|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff gemma1.1-2b-instruct-f8e4m3.uqff`|
25
+ |HQQ4|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff gemma1.1-2b-instruct-hqq4.uqff`|
26
+ |HQQ8|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff gemma1.1-2b-instruct-hqq8.uqff`|
27
+ |Q3K|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff gemma1.1-2b-instruct-q3k.uqff`|
28
+ |Q4K|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff gemma1.1-2b-instruct-q4k.uqff`|
29
+ |Q5K|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff gemma1.1-2b-instruct-q5k.uqff`|
30
+ |Q8_0|`./mistralrs-server -i plain -m EricB/gemma-1.1-2b-it-UQFF --from-uqff gemma1.1-2b-instruct-q8_0.uqff`|