The Massive Legal Embedding Benchmark (MLEB)
We present the Massive Legal Embedding Benchmark (MLEB), the largest, most
diverse, and most comprehensive open-source benchmark for legal information
retrieval to date. MLEB consists of ten expert-annotated datasets spanning
multiple jurisdictions (the US, UK, EU, Australia, Ireland, and Singapore),
document types (cases, legislation, regulatory guidance, contracts, and
literature), and task types (search, zero-shot classification, and question
answering). Seven of the datasets in MLEB were newly constructed in order to
fill domain and jurisdictional gaps in the open-source legal information
retrieval landscape. We document our methodology in building MLEB and creating
the new constituent datasets, and release our code, results, and data openly to
assist with reproducible evaluations.
68f98172b9b2e4ae04673725
Umar Butler
admin_assigned
2025-10-23T14:53:11.283Z
6497ffbf2a997a45e987e139
https://cdn-avatars.huggingface.co/v1/production/uploads/6497ffbf2a997a45e987e139/LUipMEC-wC1QeIXHSMnsx.png
Umar Butler
user
umarbutler
68f98172b9b2e4ae04673726
Abdur-Rahman Butler
claimed_verified
2025-10-23T14:51:30.656Z
63568098ef1d4c9191533b3f
https://cdn-avatars.huggingface.co/v1/production/uploads/63568098ef1d4c9191533b3f/o9IGAr_XBBiMtGRGeO3V3.png
Abdur-Rahman Butler
user
abdurrahmanbutler
68f98172b9b2e4ae04673727
Adrian Lucas Malec
admin_assigned
2025-10-23T14:53:47.869Z
65839aaab6a60815eefed378
https://cdn-avatars.huggingface.co/v1/production/uploads/65839aaab6a60815eefed378/a-Xym6IxpYy0b1B6CJpEj.jpeg
Adrian Lucas Malec
user
adlumal
The Massive Legal Embedding Benchmark (MLEB)
We present the Massive Legal Embedding Benchmark (MLEB), the largest, most
diverse, and most comprehensive open-source benchmark for legal information
retrieval to date. MLEB consists of ten expert-annotated datasets spanning
multiple jurisdictions (the US, UK, EU, Australia, Ireland, and Singapore),
document types (cases, legislation, regulatory guidance, contracts, and
literature), and task types (search, zero-shot classification, and question
answering). Seven of the datasets in MLEB were newly constructed in order to
fill domain and jurisdictional gaps in the open-source legal information
retrieval landscape. We document our methodology in building MLEB and creating
the new constituent datasets, and release our code, results, and data openly to
assist with reproducible evaluations.
MLEB is the largest open-source benchmark for legal information retrieval, encompassing multiple jurisdictions, document types, and task types.
Massive Legal Embedding Benchmark MLEB legal information retrieval expert-annotated datasets jurisdictions document types task types zero-shot classification question answering