OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training
  Tokens
We present OLMoTrace, the first system that traces the outputs of language
models back to their full, multi-trillion-token training data in real time.
OLMoTrace finds and shows verbatim matches between segments of language model
output and documents in the training text corpora. Powered by an extended
version of infini-gram (Liu et al., 2024), our system returns tracing results
within a few seconds. OLMoTrace can help users understand the behavior of
language models through the lens of their training data. We showcase how it can
be used to explore fact checking, hallucination, and the creativity of language
models. OLMoTrace is publicly available and fully open-source.
67f72bb1f9d51b79dca06d0a
Jiacheng Liu
claimed_verified
2025-04-10T06:39:44.913Z
635f46d1928a42bc95cfcf7c
https://cdn-avatars.huggingface.co/v1/production/uploads/635f46d1928a42bc95cfcf7c/5KF8aLiDCJdl7B1SdJ-7V.png
Jiacheng Liu
user
liujch1998
67f72bb1f9d51b79dca06d0b
Taylor Blanton
admin_assigned
2025-04-10T08:02:05.681Z
6675a65557208377a15f745b
/avatars/361dc6d0919f4d4545ff4fdd005332b5.svg
Taylor Blanton
user
taylorb
67f72bb1f9d51b79dca06d0c
Yanai Elazar
admin_assigned
2025-04-10T08:02:12.016Z
623ca115a795593324c4353f
/avatars/bf11fe728df2786d52ed4d2de12b48d3.svg
Yanai Elazar
user
yanaiela
67f72bb1f9d51b79dca06d0d
Sewon Min
admin_assigned
2025-04-10T08:02:17.909Z
63a76d0de27a6dbd485fe863
https://cdn-avatars.huggingface.co/v1/production/uploads/63a76d0de27a6dbd485fe863/qJJwHOuvyQGq1o0KscOF_.jpeg
Sewon Min
user
sewon
67f72bb1f9d51b79dca06d0e
YenSung Chen
admin_assigned
2025-04-10T08:02:26.950Z
6697093a37d2483826562c24
/avatars/0e526b4be6db07e2485f7ef862080339.svg
Chen
user
Yensung
67f72bb1f9d51b79dca06d0f
Arnavi Chheda-Kothary
67f72bb1f9d51b79dca06d10
Huy Tran
67f72bb1f9d51b79dca06d11
Byron Bischoff
claimed_verified
2025-04-11T07:25:58.346Z
67f85d757f56304dc72b4961
/avatars/6ae806d25632b250220145feae200058.svg
Byron Bischoff
user
byronbischoff
67f72bb1f9d51b79dca06d12
Eric Marsh
67f72bb1f9d51b79dca06d13
Michael Schmitz
67f72bb1f9d51b79dca06d14
Cassidy Trier
67f72bb1f9d51b79dca06d15
Aaron Sarnat
admin_assigned
2025-04-10T08:03:34.212Z
65b1520bf7638a13a641a620
https://cdn-avatars.huggingface.co/v1/production/uploads/65b1520bf7638a13a641a620/KTauZL0kXlmYnbkI2lFBG.png
Aaron Sarnat
user
aaronsarnat
67f72bb1f9d51b79dca06d16
Jenna James
claimed_verified
2025-04-11T07:26:00.170Z
67f7fb63ef0ad511d684f9fc
/avatars/b49d723fe735f71d7f6b809813e4ccc5.svg
James James
user
jennajames
67f72bb1f9d51b79dca06d17
Jon Borchardt
67f72bb1f9d51b79dca06d18
Bailey Kuehl
admin_assigned
2025-04-10T08:03:47.506Z
65316953791d5a2611426c20
/avatars/e632a9a30a57f62d59f9fe42eba8fd7d.svg
bailey kuehl
user
baileyk
67f72bb1f9d51b79dca06d19
Evie Cheng
67f72bb1f9d51b79dca06d1a
Karen Farley
admin_assigned
2025-04-10T08:03:58.149Z
66213c05e288b64070184cac
/avatars/ded6a173e60722200b372b8b046fc359.svg
Karen Farley
user
AI2Karen
67f72bb1f9d51b79dca06d1b
Sruthi Sreeram
67f72bb1f9d51b79dca06d1c
Taira Anderson
admin_assigned
2025-04-10T08:04:09.193Z
65de20ad4e73a7dea7fb4f08
/avatars/f3b0ad6cc9417e8ea3f0607fa62824d1.svg
Taira Anderson
user
tairaa
67f72bb1f9d51b79dca06d1d
David Albright
67f72bb1f9d51b79dca06d1e
Carissa Schoenick
admin_assigned
2025-04-10T08:04:25.492Z
6024546dc1f3c79f98e4b384
https://cdn-avatars.huggingface.co/v1/production/uploads/1612993792778-6024546dc1f3c79f98e4b384.jpeg
Carissa Schoenick
user
CarissaS
67f72bb1f9d51b79dca06d1f
Luca Soldaini
admin_assigned
2025-04-10T08:04:31.958Z
5f04d8c45d08220171a0ad32
https://cdn-avatars.huggingface.co/v1/production/uploads/5f04d8c45d08220171a0ad32/uXEta6nqBabrUlAOXnS5g.jpeg
Luca Soldaini
user
soldni
67f72bb1f9d51b79dca06d20
Dirk Groeneveld
admin_assigned
2025-04-10T08:04:40.029Z
60369745413a78f892e7339c
https://cdn-avatars.huggingface.co/v1/production/uploads/1636671879171-60369745413a78f892e7339c.png
Dirk Groeneveld
user
dirkgr
67f72bb1f9d51b79dca06d21
Rock Yuren Pang
67f72bb1f9d51b79dca06d22
Pang Wei Koh
admin_assigned
2025-04-10T08:04:53.298Z
641b4263abfce26bcf7b27de
/avatars/e91b4205e4f74b0dd8c333c23203a924.svg
Pang Wei Koh
user
pangwei
67f72bb1f9d51b79dca06d23
Noah A. Smith
67f72bb1f9d51b79dca06d24
Sophie Lebrecht
admin_assigned
2025-04-10T08:05:06.279Z
65b301f04c9e50e74a893954
/avatars/f52366959f9e7613576603c0272ff2c5.svg
Sophie Lebrecht
user
Lebrechts
67f72bb1f9d51b79dca06d25
Yejin Choi
admin_assigned
2025-04-10T08:05:13.983Z
64d42729f63b01b7f676b176
/avatars/52e54bdd6a1fb6c774a40cd70f3d7925.svg
Yejin Choi
user
yejinchoinka
67f72bb1f9d51b79dca06d26
Hannaneh Hajishirzi
67f72bb1f9d51b79dca06d27
Ali Farhadi
admin_assigned
2025-04-10T08:05:24.948Z
6660d4c1818c5c5ca0f31266
/avatars/1d2972894cb3b9df1900fdb162d9c364.svg
alifarhadi 
user
alifarhadi051
67f72bb1f9d51b79dca06d28
Jesse Dodge
admin_assigned
2025-04-10T08:05:31.942Z
6283f38567d336d3e5d5280e
/avatars/d0a54aaec74a90b050e671c191b87a80.svg
Jesse Dodge
user
JesseDodge
OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training
  Tokens
We present OLMoTrace, the first system that traces the outputs of language
models back to their full, multi-trillion-token training data in real time.
OLMoTrace finds and shows verbatim matches between segments of language model
output and documents in the training text corpora. Powered by an extended
version of infini-gram (Liu et al., 2024), our system returns tracing results
within a few seconds. OLMoTrace can help users understand the behavior of
language models through the lens of their training data. We showcase how it can
be used to explore fact checking, hallucination, and the creativity of language
models. OLMoTrace is publicly available and fully open-source.
OLMoTrace traces language model outputs to their training data in real-time, providing insights into fact checking, hallucination, and creativity through verbatim matches and infini-gram technology.
infini-gram