zELO: ELO-inspired Training Method for Rerankers and Embedding Models
We introduce a novel training methodology named zELO, which optimizes
retrieval performance via the analysis that ranking tasks are statically
equivalent to a Thurstone model. Based on the zELO method, we use unsupervised
data in order train a suite of state-of-the-art open-weight reranker models:
zerank-1 and zerank-1-small. These models achieve the highest retrieval scores
in multiple domains, including finance, legal, code, and STEM, outperforming
closed-source proprietary rerankers on both NDCG@10 and Recall. These models
also demonstrate great versatility, maintaining their 0-shot performance on
out-of-domain and private customer datasets. The training data included 112,000
queries and 100 documents per query, and was trained end-to-end from
unannotated queries and documents in less than 10,000 H100-hours.
68cb07e05a7803ff3be42c9e
Nicholas Pipitone
68cb07e05a7803ff3be42c9f
Ghita Houir Alami
68cb07e05a7803ff3be42ca0
Advaith Avadhanam
68cb07e05a7803ff3be42ca1
Anton Kaminskyi
68cb07e05a7803ff3be42ca2
Ashley Khoo
zELO: ELO-inspired Training Method for Rerankers and Embedding Models
We introduce a novel training methodology named zELO, which optimizes
retrieval performance via the analysis that ranking tasks are statically
equivalent to a Thurstone model. Based on the zELO method, we use unsupervised
data in order train a suite of state-of-the-art open-weight reranker models:
zerank-1 and zerank-1-small. These models achieve the highest retrieval scores
in multiple domains, including finance, legal, code, and STEM, outperforming
closed-source proprietary rerankers on both NDCG@10 and Recall. These models
also demonstrate great versatility, maintaining their 0-shot performance on
out-of-domain and private customer datasets. The training data included 112,000
queries and 100 documents per query, and was trained end-to-end from
unannotated queries and documents in less than 10,000 H100-hours.
A novel training methodology named zELO optimizes retrieval performance by treating ranking tasks as equivalent to a Thurstone model, resulting in state-of-the-art open-weight reranker models that outperform proprietary models across various domains.
zELO Thurstone model reranker models NDCG@10 Recall 0-shot performance unannotated queries documents H100-hours