Update README.md
Browse files
README.md
CHANGED
|
@@ -188,6 +188,18 @@ The model was evaluated on the OpenLLM leaderboard task, using [lm-evaluation-ha
|
|
| 188 |
--greedy
|
| 189 |
```
|
| 190 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 191 |
</details>
|
| 192 |
|
| 193 |
|
|
@@ -297,6 +309,18 @@ The model was evaluated on the OpenLLM leaderboard task, using [lm-evaluation-ha
|
|
| 297 |
<td><b>61.28</b></td>
|
| 298 |
<td><b>100.15</b></td>
|
| 299 |
</tr>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 300 |
<!-- <td rowspan="4" ><strong>Coding</strong>
|
| 301 |
</td>
|
| 302 |
<td>HumanEval pass@1
|
|
|
|
| 188 |
--greedy
|
| 189 |
```
|
| 190 |
|
| 191 |
+
|
| 192 |
+
**Multimodal Evaluation**
|
| 193 |
+
```
|
| 194 |
+
lm_eval \
|
| 195 |
+
--model vllm-vlm \
|
| 196 |
+
--model_args pretrained="RedHatAI/Llama-4-Maverick-17B-128E-Instruct-FP8-block",dtype=auto,add_bos_token=False,max_model_len=1000000,tensor_parallel_size=8,gpu_memory_utilization=0.9,enable_chunked_prefill=True,trust_remote_code=True,max_images=10 \
|
| 197 |
+
--tasks mmlu \
|
| 198 |
+
--apply_chat_template \
|
| 199 |
+
--batch_size auto
|
| 200 |
+
```
|
| 201 |
+
|
| 202 |
+
|
| 203 |
</details>
|
| 204 |
|
| 205 |
|
|
|
|
| 309 |
<td><b>61.28</b></td>
|
| 310 |
<td><b>100.15</b></td>
|
| 311 |
</tr>
|
| 312 |
+
<tr>
|
| 313 |
+
<td rowspan="6" ><strong>Multi-modal</strong>
|
| 314 |
+
</td>
|
| 315 |
+
<td>MMMU (val)
|
| 316 |
+
</td>
|
| 317 |
+
<td>79.08
|
| 318 |
+
</td>
|
| 319 |
+
<td>78.50
|
| 320 |
+
</td>
|
| 321 |
+
<td>99.26
|
| 322 |
+
</td>
|
| 323 |
+
</tr>
|
| 324 |
<!-- <td rowspan="4" ><strong>Coding</strong>
|
| 325 |
</td>
|
| 326 |
<td>HumanEval pass@1
|