Introducing the Open Chain of Thought Leaderboard
•
37
AI & ML interests
Building breatkthrough AI to solve the world's biggest problems.
Recent Activity
View all activity
Papers
olmOCR 2: Unit Test Rewards for Document OCR
MolmoAct: Action Reasoning Models that can Reason in Space
Articles
OlmoEarth pre-trained and fine-tuned models
All models for the MolmoAct (Multimodal Open Language Model for Action) release.
Datasets, spaces, and models for Reward Bench 2 benchmark and paper!
olmOCR is a document recognition pipeline for efficiently converting documents into plain text.
olmocr.allenai.org
Improved OLMoE for iOS app. Read more: https://allenai.org/blog/olmoe-app
All models released with Tulu 3 -- state of the art open post-training recipes.
Artifacts for open multimodal language models.
Artifacts for the first set of OLMo models.
Datasets, spaces, and models for the reward model benchmark!
The set of models associated with the paper "Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2"
Data and models to enhance instruction-following for scientific literature understanding.
ZebraLogic Bench: Testing the Limits of LLMs in Logical Reasoning
-
90
Zebra Logic Bench
🦓Display and explore a leaderboard for model evaluations
-
allenai/ZebraLogicBench
Viewer • Updated • 4.26k • 2.28k • 22 -
allenai/ZebraLogicBench-private
Viewer • Updated • 4.26k • 854 • 11 -
Faith and Fate: Limits of Transformers on Compositionality
Paper • 2305.18654 • Published • 7
Ai2 Climate Emulator (ACE) is a family of fast ML models that simulate global atmospheric variability over time scales ranging from hours to centuries
All datasets for the MolmoAct (Multimodal Open Language Model for Action) release.
Datasets for IFBench benchmark and paper!
Artifacts for the OLMo 2 release.
A suite of models, data, and evals over 25 corpora, 14 sizes, and 3 seeds to measure how accurately small experiments predict rankings at large scale.
A set of vision-language datasets built by Ai2 and used to train the Molmo family of models. Read more at https://molmo.allenai.org/blog
All datasets released with Tulu 3 -- state of the art open post-training recipes.
-
allenai/tulu-3-sft-mixture
Viewer • Updated • 939k • 14.3k • 183 -
allenai/llama-3.1-tulu-3-8b-preference-mixture
Viewer • Updated • 273k • 2.3k • 23 -
allenai/llama-3.1-tulu-3-70b-preference-mixture
Viewer • Updated • 337k • 120 • 19 -
allenai/llama-3.1-tulu-3-405b-preference-mixture
Viewer • Updated • 361k • 273 • 6
Artifacts for open mixture-of-experts language models.
A suite of models trained using DPO and PPO across a wide variety (up to 14) of preference datasets. See https://arxiv.org/abs/2406.09279 for more!
Dataset and baseline models for Paloma, a benchmark of language model fit to 546 textual domains
-
231
AI2 WildBench Leaderboard (V2)
🦁Display and explore a leaderboard of language models
-
allenai/WildBench
Viewer • Updated • 2.3k • 1.28k • 37 -
allenai/WildBench-V2-Model-Outputs
Viewer • Updated • 62.5k • 2.24k • 2 -
WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild
Paper • 2406.04770 • Published • 30
Safety data, moderation tools and safe LLMs.
These model's tokenizer did not use HF's fast tokenizer, resulting in variations in how pre-tokenization was applied. Resolved in latest versions.
-
allenai/OLMo-2-1124-13B-Instruct-preview
Text Generation • 14B • Updated • 193 • 57 -
allenai/OLMo-2-1124-7B-Instruct-preview
Text Generation • 7B • Updated • 62 • 47 -
allenai/OLMo-2-1124-7B-SFT-Preview
Text Generation • Updated • 93 • 3 -
allenai/OLMo-2-1124-7B-DPO-Preview
Text Generation • Updated • 79 • 2
OlmoEarth pre-trained and fine-tuned models
All datasets for the MolmoAct (Multimodal Open Language Model for Action) release.
All models for the MolmoAct (Multimodal Open Language Model for Action) release.
Datasets for IFBench benchmark and paper!
Datasets, spaces, and models for Reward Bench 2 benchmark and paper!
Artifacts for the OLMo 2 release.
olmOCR is a document recognition pipeline for efficiently converting documents into plain text.
olmocr.allenai.org
A suite of models, data, and evals over 25 corpora, 14 sizes, and 3 seeds to measure how accurately small experiments predict rankings at large scale.
Improved OLMoE for iOS app. Read more: https://allenai.org/blog/olmoe-app
A set of vision-language datasets built by Ai2 and used to train the Molmo family of models. Read more at https://molmo.allenai.org/blog
All models released with Tulu 3 -- state of the art open post-training recipes.
All datasets released with Tulu 3 -- state of the art open post-training recipes.
-
allenai/tulu-3-sft-mixture
Viewer • Updated • 939k • 14.3k • 183 -
allenai/llama-3.1-tulu-3-8b-preference-mixture
Viewer • Updated • 273k • 2.3k • 23 -
allenai/llama-3.1-tulu-3-70b-preference-mixture
Viewer • Updated • 337k • 120 • 19 -
allenai/llama-3.1-tulu-3-405b-preference-mixture
Viewer • Updated • 361k • 273 • 6
Artifacts for open multimodal language models.
Artifacts for open mixture-of-experts language models.
Artifacts for the first set of OLMo models.
A suite of models trained using DPO and PPO across a wide variety (up to 14) of preference datasets. See https://arxiv.org/abs/2406.09279 for more!
Datasets, spaces, and models for the reward model benchmark!
Dataset and baseline models for Paloma, a benchmark of language model fit to 546 textual domains
The set of models associated with the paper "Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2"
-
231
AI2 WildBench Leaderboard (V2)
🦁Display and explore a leaderboard of language models
-
allenai/WildBench
Viewer • Updated • 2.3k • 1.28k • 37 -
allenai/WildBench-V2-Model-Outputs
Viewer • Updated • 62.5k • 2.24k • 2 -
WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild
Paper • 2406.04770 • Published • 30
Data and models to enhance instruction-following for scientific literature understanding.
Safety data, moderation tools and safe LLMs.
ZebraLogic Bench: Testing the Limits of LLMs in Logical Reasoning
-
90
Zebra Logic Bench
🦓Display and explore a leaderboard for model evaluations
-
allenai/ZebraLogicBench
Viewer • Updated • 4.26k • 2.28k • 22 -
allenai/ZebraLogicBench-private
Viewer • Updated • 4.26k • 854 • 11 -
Faith and Fate: Limits of Transformers on Compositionality
Paper • 2305.18654 • Published • 7
These model's tokenizer did not use HF's fast tokenizer, resulting in variations in how pre-tokenization was applied. Resolved in latest versions.
-
allenai/OLMo-2-1124-13B-Instruct-preview
Text Generation • 14B • Updated • 193 • 57 -
allenai/OLMo-2-1124-7B-Instruct-preview
Text Generation • 7B • Updated • 62 • 47 -
allenai/OLMo-2-1124-7B-SFT-Preview
Text Generation • Updated • 93 • 3 -
allenai/OLMo-2-1124-7B-DPO-Preview
Text Generation • Updated • 79 • 2
Ai2 Climate Emulator (ACE) is a family of fast ML models that simulate global atmospheric variability over time scales ranging from hours to centuries