Granther
/

Gemma-2-9B-Instruct-4Bit-GPTQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions

Gemma-2-9B-Instruct-4Bit-GPTQ

Original Model: gemma-2-9b-it
Model Creator: google

Quantization

This model was quantized with the Auto-GPTQ library

Metrics

Benchmark	Metric	Gemma 2 GPTQ	Gemma 2 9B it
PIQA	0-shot	80.52	80.79
MMLU	5-shot	52.0	50.00

Downloads last month: 18

Safetensors

Model size

2B params

Tensor type

I32

·

F16

·

Model tree for Granther/Gemma-2-9B-Instruct-4Bit-GPTQ

Base model

google/gemma-2-9b

Finetuned

google/gemma-2-9b-it

Quantized

(150)

this model