Update README.md

README.md

@@ -3,6 +3,7 @@ license: apache-2.0
 datasets:
 - allenai/dolma
 - allenai/tulu-v2-sft-mixture
+- allenai/ultrafeedback_binarized_cleaned
 language:
 - en
 ---
@@ -11,7 +12,7 @@ language:
 <img src="https://allenai.org/olmo/olmo-7b-animation.gif" alt="OLMo Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
 
 
-# Model Card for OLMo 1.7 7B Nitro SFT
+# Model Card for OLMo 7B July 2024 SFT
 
 **Requires transformers versions v4.40.0 or newer**
 
@@ -20,7 +21,8 @@ OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the scie
 The OLMo base models are trained on the [Dolma](https://huggingface.co/datasets/allenai/dolma) dataset.
 The adapted versions are trained on the [Tulu SFT mixture](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) and, for the Instruct version, a [cleaned version of the UltraFeedback dataset](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned).
 
-OLMo 1.7 Nitro 7B Instruct and OLMo SFT are two adapted versions of these models trained for better question answering.
+OLMo 7B Instruct SFT are two adapted versions of these models trained for better question answering.
+These are updated OLMo models corresponding to our July 2024 release.
 They show the performance gain that OLMo base models can achieve with existing fine-tuning techniques.
 
 ## Model Details
@@ -28,13 +30,13 @@ They show the performance gain that OLMo base models can achieve with existing f
 We release two adapted model versions:
 | Model | Training Method(s) | Datasets | Context Length | 
 |------|--------|---------|--|
-| [OLMo 1.7 7B Nitro SFT](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-SFT-hf)   | SFT | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture)     | 2048        | 
-| [OLMo 1.7 7B Nitro Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-Instruct-hf) | SFT + DPO   | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) + [Ultrafeedback Cleaned](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned)     | 2048 |
+| [OLMo 7B July 2024 SFT](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-SFT-hf)   | SFT | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture)     | 2048        | 
+| [OLMo 7B July 2024 Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-Instruct-hf) | SFT + DPO   | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) + [Ultrafeedback Cleaned](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned)     | 2048 |
 
-These models are both trained on top of OLMo 1.7 7b 'Nitro':
+These models are both trained on top of OLMo 7b July 2024:
 | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length |
 |------|--------|---------|-------------|-----------------|----------------|
-| [OLMo 1.7 7B](https://huggingface.co/allenai/OLMo-1.7-7B-hf)   | 2.7T |32     | 4096        | 32              | 4096  |
+| [OLMo 7B July 2024](https://huggingface.co/allenai/OLMo-1.7-7B-hf)   | 2.7T |32     | 4096        | 32              | 4096  |
 
 
 ### Model Description
@@ -66,8 +68,8 @@ You can run these models using recent (>= 4.40) versions of transformers.
 
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
-olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-1.7-7B-Nitro-SFT-hf")
-tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-1.7-7B-Nitro-SFT-hf")
+olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-7B-0724-Instruct-hf")
+tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-7B-0724-Instruct-hf")
 chat = [
     { "role": "user", "content": "What is language modeling?" },
 ]
@@ -93,9 +95,9 @@ Core model results for the 7B adapted models are found below.
 
 | Model                 | MMLU 0-shot ↑ | AlpacaEval %win ↑ | ToxiGen % Toxic ↓ | TruthfulQA %Info+True ↑ |
 |-----------------------|---------------|--------------------|--------------------|-------------------------|
-| **OLMo 1.7 'Nitro' base**      | 50.8          | -                  | 85.2               | 28.4                    |
-| **[OLMo 1.7 7B Nitro SFT](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-SFT-hf)**         |    54.2       | 70.9               | .1               | 44.4                    |
-| **[OLMo 1.7 7B Nitro Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-Instruct-hf)**     | 52.8          | 83.5               | 1.7                | 70.3                    |
+| **OLMo July 2024 base**      | 50.8          | -                  | 85.2               | 28.4                    |
+| **[OLMo 7B July 2024 SFT](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-SFT-hf)**         |    54.2       | 70.9               | .1               | 44.4                    |
+| **[OLMo 7B July 2024 Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-Instruct-hf)**     | 52.8          | 83.5               | 1.7                | 70.3                    |