Create README.md

README.md

@@ -0,0 +1,47 @@
+---
+license: apache-2.0
+datasets:
+- shibing624/alpaca-zh
+language:
+- zh
+tags:
+- LoRA
+- LLaMA
+- Alpaca
+- PEFT
+- int8
+---
+
+# Model Card for llama-7b-alpaca-zh-20k
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+```python
+from peft import PeftModel
+from transformers import GenerationConfig, LlamaForCausalLM, LlamaTokenizer
+
+
+max_memory = {i: "15GIB" for i in range(torch.cuda.device_count())}
+tokenizer = LlamaTokenizer.from_pretrained(base_model)
+model = LlamaForCausalLM.from_pretrained(
+    base_model,
+    load_in_8bit=True,
+    torch_dtype=torch.float16,
+    device_map="auto"
+    max_memory=max_memory
+)
+model = PeftModel.from_pretrained(
+    model,
+    lora_weights,
+    torch_dtype=torch.float16,
+    max_memory=max_memory
+)
+```