Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
Missing Tasks?
#157
by
Chao985
- opened
I'd like to ask if all the tasks listed in the leaderboard can be evaluated using the MTEB library?
I noticed that many tasks are not included in the mteb --available_tasks list.
For instance, the German retrieval task GerDaLIR, the French recall task AlloprofRetrieval, and the BSARDRetrieval task.
It seems that these tasks cannot be evaluated using the MTEB library.
Could you possibly provide more evaluation code examples?
All tasks are presented in mteb and can be evaluated using mteb.
mteb available_tasks | grep AlloprofRetrieval
- AlloprofRetrieval, s2p
mteb available_tasks | grep GerDaLIR
- GerDaLIR, s2p
- GerDaLIRSmall, p2p
Btw. We don’t take issues here, but over on the mteb GitHub so if you want to follow up you will have to do it there.