iproskurina/pythia-6.9b-awq-int4

pythia-6.9b - AWQ (4-bit)

Source model: EleutherAI/pythia-6.9b

This model was quantized to 4-bit using VLLM-Compressor.

Quantization parameters: 4-bit, symmetric scheme.

Usage

# pip install vllm
from vllm import LLM
model = LLM("iproskurina/pythia-6.9b-awq-int4")
output = model.generate("The capital of France is")
print(output)```