Spaces:
Running
Running
update instructions
Browse files
utils.py
CHANGED
|
@@ -99,13 +99,17 @@ SUBMIT_INTRODUCTION = """# Submit on MMEB Leaderboard Introduction \n
|
|
| 99 |
}
|
| 100 |
}
|
| 101 |
```
|
| 102 |
-
3. **Step 3️⃣:** Finally, create a pull request and upload the generated JSON file to the ***scores*** folder
|
| 103 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
## ⚠️ Special Instructions for submitting to MMEB Image (Previously MMEB-V1) Leaderboard
|
| 105 |
We understand that some researchers want to exclusively submit to the Image leaderboard, but unfortunately our current leaderboard system cannot exclude your model from other modalities' leaderboards.
|
| 106 |
To do so, please run the 36 image datasets only and simply ignore the other datasets.
|
| 107 |
The leaderboard will automatically assign a 0 to the missing datasets.
|
| 108 |
-
We might be able to hide your model from the other leaderboards in the next leaderboard updates, but for now your model will be shown on all leaderboards and might have a lower rank.
|
| 109 |
We highly recommend joining our [discord server](https://discord.gg/njyKubdtry), which provides a convenient way to stay informed with latest updates, or share any feedback you have for improving the leaderboard experience. We appreciate your contributions to the MMEB community!
|
| 110 |
"""
|
| 111 |
|
|
|
|
| 99 |
}
|
| 100 |
}
|
| 101 |
```
|
| 102 |
+
3. **Step 3️⃣:** Finally, create a pull request and upload the generated JSON file to the ***scores*** folder.
|
| 103 |
+
- You can directly use the Hugging Face Space's web UI to upload your files:
|
| 104 |
+
- Go to the [scores folder](https://huggingface.co/spaces/TIGER-Lab/MMEB-Leaderboard/upload/main/scores)
|
| 105 |
+
- Select "Upload file" and upload your JSON files.
|
| 106 |
+
- Please inform us on [our discord server](https://discord.gg/njyKubdtry), or send us an email at [email protected], or leave a simple message (simply @ us) in the PR to indicate that you are ready to merge your PR.
|
| 107 |
+
- We will then review your submission and update the leaderboard accordingly. \n\n
|
| 108 |
## ⚠️ Special Instructions for submitting to MMEB Image (Previously MMEB-V1) Leaderboard
|
| 109 |
We understand that some researchers want to exclusively submit to the Image leaderboard, but unfortunately our current leaderboard system cannot exclude your model from other modalities' leaderboards.
|
| 110 |
To do so, please run the 36 image datasets only and simply ignore the other datasets.
|
| 111 |
The leaderboard will automatically assign a 0 to the missing datasets.
|
| 112 |
+
We might be able to hide your model from the other leaderboards in the next leaderboard updates, but for now your model will be shown on all leaderboards and might have a lower rank. \n
|
| 113 |
We highly recommend joining our [discord server](https://discord.gg/njyKubdtry), which provides a convenient way to stay informed with latest updates, or share any feedback you have for improving the leaderboard experience. We appreciate your contributions to the MMEB community!
|
| 114 |
"""
|
| 115 |
|