Nomic Embed: Training a Reproducible Long Context Text Embedder
This technical report describes the training of nomic-embed-text-v1, the
first fully reproducible, open-source, open-weights, open-data, 8192 context
length English text embedding model that outperforms both OpenAI Ada-002 and
OpenAI text-embedding-3-small on short and long-context tasks. We release the
training code and model weights under an Apache 2 license. In contrast with
other open-source models, we release a training data loader with 235 million
curated text pairs that allows for the full replication of nomic-embed-text-v1.
You can find code and data to replicate the model at
https://github.com/nomic-ai/contrastors
65c071a277b1ecb76d6a3c5f
Zach Nussbaum
admin_assigned
2024-02-05T12:22:07.728Z
607997c83a565c15675055b3
https://cdn-avatars.huggingface.co/v1/production/uploads/607997c83a565c15675055b3/KCVb16r2WHSqyRbUyp4eK.jpeg
Zach Nussbaum
user
zpn
65c071a277b1ecb76d6a3c60
John X. Morris
65c071a277b1ecb76d6a3c61
Brandon Duderstadt
extracted_confirmed
2024-02-05T14:52:27.069Z
6421f12aeaad1bcb28b22bb4
/avatars/68b9f9a8a5d7b3b951932527deab0275.svg
Brandon Duderstadt
user
bstadt
65c071a277b1ecb76d6a3c62
Andriy Mulyar
extracted_pending
2024-02-05T05:26:59.654Z
641f01fe6d51620635e118e9
/avatars/1c251ef5c571c2d6f5eec212f02ea26c.svg
Andriy Mulyar
user
andriym
Nomic Embed: Training a Reproducible Long Context Text Embedder
This technical report describes the training of nomic-embed-text-v1, the
first fully reproducible, open-source, open-weights, open-data, 8192 context
length English text embedding model that outperforms both OpenAI Ada-002 and
OpenAI text-embedding-3-small on short and long-context tasks. We release the
training code and model weights under an Apache 2 license. In contrast with
other open-source models, we release a training data loader with 235 million
curated text pairs that allows for the full replication of nomic-embed-text-v1.
You can find code and data to replicate the model at
https://github.com/nomic-ai/contrastors
The report details the development and release of nomic-embed-text-v1, an open-source English text embedding model that surpasses OpenAI models in both short and long-context tasks.