Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
score update
Browse files- scores/Crotchet-embedding-2B.json +2 -2
- utils.py +2 -0
scores/Crotchet-embedding-2B.json
CHANGED
|
@@ -2087,7 +2087,7 @@
|
|
| 2087 |
"num_pred": 863,
|
| 2088 |
"num_data": 863
|
| 2089 |
},
|
| 2090 |
-
"ViDoSeek-page": {
|
| 2091 |
"hit@1": 0.6304728546409807,
|
| 2092 |
"hit@5": 0.9264448336252189,
|
| 2093 |
"hit@10": 0.9728546409807356,
|
|
@@ -2143,7 +2143,7 @@
|
|
| 2143 |
"num_pred": 1142,
|
| 2144 |
"num_data": 1142
|
| 2145 |
},
|
| 2146 |
-
"MMLongBench-page": {
|
| 2147 |
"hit@1": 0.4522673031026253,
|
| 2148 |
"hit@5": 0.6599045346062052,
|
| 2149 |
"hit@10": 0.7195704057279236,
|
|
|
|
| 2087 |
"num_pred": 863,
|
| 2088 |
"num_data": 863
|
| 2089 |
},
|
| 2090 |
+
"ViDoSeek-page-fixed": {
|
| 2091 |
"hit@1": 0.6304728546409807,
|
| 2092 |
"hit@5": 0.9264448336252189,
|
| 2093 |
"hit@10": 0.9728546409807356,
|
|
|
|
| 2143 |
"num_pred": 1142,
|
| 2144 |
"num_data": 1142
|
| 2145 |
},
|
| 2146 |
+
"MMLongBench-page-fixed": {
|
| 2147 |
"hit@1": 0.4522673031026253,
|
| 2148 |
"hit@5": 0.6599045346062052,
|
| 2149 |
"hit@10": 0.7195704057279236,
|
utils.py
CHANGED
|
@@ -37,6 +37,8 @@ The scores submitted before the fix have been renamed to **ViDoSeek-page-before-
|
|
| 37 |
|
| 38 |
Here is the list of models affected by this fix: [File](https://huggingface.co/spaces/TIGER-Lab/MMEB-Leaderboard/blob/main/archive/cached_vd_scores_before_fix.jsonl). If your model is on this list, we kindly ask you to visit the VisDoc leaderboard and verify if your model is missing the fixed scores, and then re-evaluate your model on these two datasets and submit the updated scores to ensure accurate ranking on the leaderboard.
|
| 39 |
|
|
|
|
|
|
|
| 40 |
Please let us know if you have any questions or concerns, or there is an issue with your model's scores. We appreciate your understanding and cooperation in maintaining the integrity of the benchmark.
|
| 41 |
**===========================================================================================================**
|
| 42 |
|
|
|
|
| 37 |
|
| 38 |
Here is the list of models affected by this fix: [File](https://huggingface.co/spaces/TIGER-Lab/MMEB-Leaderboard/blob/main/archive/cached_vd_scores_before_fix.jsonl). If your model is on this list, we kindly ask you to visit the VisDoc leaderboard and verify if your model is missing the fixed scores, and then re-evaluate your model on these two datasets and submit the updated scores to ensure accurate ranking on the leaderboard.
|
| 39 |
|
| 40 |
+
In the generated report sheet, please double check that ViDoSeek-page and MMLongBench-page have been renamed to ViDoSeek-page-fixed and MMLongBench-page-fixed respectively.
|
| 41 |
+
|
| 42 |
Please let us know if you have any questions or concerns, or there is an issue with your model's scores. We appreciate your understanding and cooperation in maintaining the integrity of the benchmark.
|
| 43 |
**===========================================================================================================**
|
| 44 |
|