Add ZipNN stuff

README.md

@@ -12,8 +12,67 @@ widget:
   - role: user
     content: Can you provide ways to eat combinations of bananas and dragonfruits?
 library_name: transformers
+base_model:
+- microsoft/Phi-3.5-mini-instruct
 ---
 
+# Disclaimer and Requirements
+
+This model is a clone of [**microsoft/Phi-3.5-mini-instruct**](https://huggingface.co/microsoft/Phi-3.5-mini-instruct) compressed using ZipNN. Compressed losslessly to 67% its original size, ZipNN saved ~3GB in storage and potentially ~1PB in data transfer **monthly**.
+
+### Requirement
+
+In order to use the model, ZipNN is necessary:
+```bash
+pip install zipnn
+```
+### Use This Model
+```python
+# Use a pipeline as a high-level helper
+from transformers import pipeline
+from zipnn import zipnn_hf
+
+zipnn_hf()
+
+messages = [
+    {"role": "user", "content": "Who are you?"},
+]
+pipe = pipeline("text-generation", model="royleibov/Phi-3.5-mini-instruct-ZipNN-Compressed")
+pipe(messages)
+```
+```python
+# Load model directly
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from zipnn import zipnn_hf
+
+zipnn_hf()
+
+torch.random.manual_seed(0)
+
+model = AutoModelForCausalLM.from_pretrained(
+    "royleibov/Phi-3.5-mini-instruct-ZipNN-Compressed", 
+    device_map="cuda", 
+    torch_dtype="auto", 
+    trust_remote_code=True, 
+)
+tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3.5-mini-instruct")
+```
+### ZipNN
+ZipNN also allows you to seemlessly save local disk space in your cache after the model is downloaded.
+
+To compress the cached model, simply run:
+```bash
+python zipnn_compress_path.py safetensors --model royleibov/Phi-3.5-mini-instruct-ZipNN-Compressed --hf_cache
+```
+
+The model will be decompressed automatically and safely as long as `zipnn_hf()` is added at the top of the file like in the [example above](#use-this-model).
+
+To decompress manualy, simply run:
+```bash
+python zipnn_decompress_path.py --model royleibov/Phi-3.5-mini-instruct-ZipNN-Compressed --hf_cache
+```
+
 ## Model Summary
 
 Phi-3.5-mini is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available websites - with a focus on very high-quality, reasoning dense data. The model belongs to the Phi-3 model family and supports 128K token context length. The model underwent a rigorous enhancement process, incorporating both supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures.
@@ -141,16 +200,19 @@ After obtaining the Phi-3.5-mini-instruct model checkpoint, users can use this s
 ```python
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
+from zipnn import zipnn_hf
+
+zipnn_hf()
 
 torch.random.manual_seed(0)
 
 model = AutoModelForCausalLM.from_pretrained(
-    "microsoft/Phi-3.5-mini-instruct", 
+    "royleibov/Phi-3.5-mini-instruct-ZipNN-Compressed", 
     device_map="cuda", 
     torch_dtype="auto", 
     trust_remote_code=True, 
 )
-tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3.5-mini-instruct")
+tokenizer = AutoTokenizer.from_pretrained("royleibov/Phi-3.5-mini-instruct-ZipNN-Compressed")
 
 messages = [
     {"role": "system", "content": "You are a helpful AI assistant."},