Update README.md
Browse files
README.md
CHANGED
|
@@ -10,7 +10,14 @@ pipeline_tag: text-generation
|
|
| 10 |
base_model: meta-llama/Llama-2-13b-hf
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
This instruction model was built via parameter-efficient QLoRA finetuning of [llama-2-13b](https://huggingface.co/meta-llama/Llama-2-13b-hf) on the first 100k rows of [ehartford/dolphin](https://huggingface.co/datasets/ehartford/dolphin) (an open-source implementation of [Microsoft's Orca](https://www.microsoft.com/en-us/research/publication/orca-progressive-learning-from-complex-explanation-traces-of-gpt-4/)). Finetuning was executed on a single A6000 (48 GB) for roughly 18 hours on the [Lambda Labs](https://cloud.lambdalabs.com/instances) platform.
|
| 16 |
|
|
@@ -30,9 +37,9 @@ We use state-of-the-art [Language Model Evaluation Harness](https://github.com/E
|
|
| 30 |
|
| 31 |
* Model license: Llama 2 Community License Agreement
|
| 32 |
* Basic usage: [notebook](assets/basic_inference_llama_2_13b_dolphin.ipynb)
|
| 33 |
-
* Finetuning script: [script](https://github.com/daniel-furman/sft-demos/blob/main/src/sft/one_gpu/llama-2/dolphin/sft-
|
| 34 |
-
* Loss curves: [plot](https://huggingface.co/dfurman/
|
| 35 |
-
* Runtime stats: [table](https://huggingface.co/dfurman/
|
| 36 |
|
| 37 |
## Example prompts and responses
|
| 38 |
|
|
@@ -41,7 +48,7 @@ Example 1:
|
|
| 41 |
**User**:
|
| 42 |
>You are a helpful assistant. Write me a numbered list of things to do in New York City.\n
|
| 43 |
|
| 44 |
-
**
|
| 45 |
|
| 46 |
> 1. Visit the iconic Statue of Liberty and Ellis Island.
|
| 47 |
> 2. Take a stroll through Central Park and enjoy its many attractions.
|
|
@@ -61,7 +68,7 @@ Example 2:
|
|
| 61 |
**User**:
|
| 62 |
>You are a helpful assistant. Write a short email inviting my friends to a dinner party on Friday. Respond succinctly.\n"
|
| 63 |
|
| 64 |
-
**
|
| 65 |
|
| 66 |
> Subject: Friday Dinner Party Invitation
|
| 67 |
>
|
|
@@ -97,7 +104,7 @@ The llama-2-13b models have been modified from a standard transformer in the fol
|
|
| 97 |
|
| 98 |
This model was trained on a single A6000 (48 GB) for about 18 hours using the [Lambda Labs](https://cloud.lambdalabs.com/instances) platform.
|
| 99 |
|
| 100 |
-

|
|
| 138 |
```
|
| 139 |
|
| 140 |
```python
|
| 141 |
-
peft_model_id = "dfurman/
|
| 142 |
config = PeftConfig.from_pretrained(peft_model_id)
|
| 143 |
|
| 144 |
bnb_config = BitsAndBytesConfig(
|
|
@@ -194,7 +201,7 @@ print(tokenizer.decode(output["sequences"][0], skip_special_tokens=True))
|
|
| 194 |
| 2.93 | 1x A100 (40 GB SXM) | torch | bfloat16 | 25 |
|
| 195 |
| 3.24 | 1x A6000 (48 GB) | torch | bfloat16 | 25 |
|
| 196 |
|
| 197 |
-
The above runtime stats were generated from this [notebook](https://github.com/daniel-furman/sft-demos/blob/main/src/sft/one_gpu/llama-2/dolphin/postprocessing-
|
| 198 |
|
| 199 |
## Acknowledgements
|
| 200 |
|
|
|
|
| 10 |
base_model: meta-llama/Llama-2-13b-hf
|
| 11 |
---
|
| 12 |
|
| 13 |
+
<div align="center">
|
| 14 |
+
|
| 15 |
+
<img src="./assets/llama.png" width="150px">
|
| 16 |
+
|
| 17 |
+
</div>
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
# Llama-2-13B-Instruct-v0.1 🦙🐬
|
| 21 |
|
| 22 |
This instruction model was built via parameter-efficient QLoRA finetuning of [llama-2-13b](https://huggingface.co/meta-llama/Llama-2-13b-hf) on the first 100k rows of [ehartford/dolphin](https://huggingface.co/datasets/ehartford/dolphin) (an open-source implementation of [Microsoft's Orca](https://www.microsoft.com/en-us/research/publication/orca-progressive-learning-from-complex-explanation-traces-of-gpt-4/)). Finetuning was executed on a single A6000 (48 GB) for roughly 18 hours on the [Lambda Labs](https://cloud.lambdalabs.com/instances) platform.
|
| 23 |
|
|
|
|
| 37 |
|
| 38 |
* Model license: Llama 2 Community License Agreement
|
| 39 |
* Basic usage: [notebook](assets/basic_inference_llama_2_13b_dolphin.ipynb)
|
| 40 |
+
* Finetuning script: [script](https://github.com/daniel-furman/sft-demos/blob/main/src/sft/one_gpu/llama-2/dolphin/sft-Llama-2-13B-Instruct-v0.1-peft.py)
|
| 41 |
+
* Loss curves: [plot](https://huggingface.co/dfurman/Llama-2-13B-Instruct-v0.1-peft#finetuning-description)
|
| 42 |
+
* Runtime stats: [table](https://huggingface.co/dfurman/Llama-2-13B-Instruct-v0.1-peft#runtime-tests)
|
| 43 |
|
| 44 |
## Example prompts and responses
|
| 45 |
|
|
|
|
| 48 |
**User**:
|
| 49 |
>You are a helpful assistant. Write me a numbered list of things to do in New York City.\n
|
| 50 |
|
| 51 |
+
**Llama-2-13B-Instruct-v0.1-peft**:
|
| 52 |
|
| 53 |
> 1. Visit the iconic Statue of Liberty and Ellis Island.
|
| 54 |
> 2. Take a stroll through Central Park and enjoy its many attractions.
|
|
|
|
| 68 |
**User**:
|
| 69 |
>You are a helpful assistant. Write a short email inviting my friends to a dinner party on Friday. Respond succinctly.\n"
|
| 70 |
|
| 71 |
+
**Llama-2-13B-Instruct-v0.1-peft**:
|
| 72 |
|
| 73 |
> Subject: Friday Dinner Party Invitation
|
| 74 |
>
|
|
|
|
| 104 |
|
| 105 |
This model was trained on a single A6000 (48 GB) for about 18 hours using the [Lambda Labs](https://cloud.lambdalabs.com/instances) platform.
|
| 106 |
|
| 107 |
+

|
| 108 |
|
| 109 |
The above loss curve was generated from the run's private wandb.ai log.
|
| 110 |
|
|
|
|
| 145 |
```
|
| 146 |
|
| 147 |
```python
|
| 148 |
+
peft_model_id = "dfurman/Llama-2-13B-Instruct-v0.1-peft"
|
| 149 |
config = PeftConfig.from_pretrained(peft_model_id)
|
| 150 |
|
| 151 |
bnb_config = BitsAndBytesConfig(
|
|
|
|
| 201 |
| 2.93 | 1x A100 (40 GB SXM) | torch | bfloat16 | 25 |
|
| 202 |
| 3.24 | 1x A6000 (48 GB) | torch | bfloat16 | 25 |
|
| 203 |
|
| 204 |
+
The above runtime stats were generated from this [notebook](https://github.com/daniel-furman/sft-demos/blob/main/src/sft/one_gpu/llama-2/dolphin/postprocessing-Llama-2-13B-Instruct-v0.1-peft.ipynb).
|
| 205 |
|
| 206 |
## Acknowledgements
|
| 207 |
|