MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in
  Video Scenarios
Multimodal Large Language Models (MLLMs) have achieved considerable accuracy
in Optical Character Recognition (OCR) from static images. However, their
efficacy in video OCR is significantly diminished due to factors such as motion
blur, temporal variations, and visual effects inherent in video content. To
provide clearer guidance for training practical MLLMs, we introduce the
MME-VideoOCR benchmark, which encompasses a comprehensive range of video OCR
application scenarios. MME-VideoOCR features 10 task categories comprising 25
individual tasks and spans 44 diverse scenarios. These tasks extend beyond text
recognition to incorporate deeper comprehension and reasoning of textual
content within videos. The benchmark consists of 1,464 videos with varying
resolutions, aspect ratios, and durations, along with 2,000 meticulously
curated, manually annotated question-answer pairs. We evaluate 18
state-of-the-art MLLMs on MME-VideoOCR, revealing that even the best-performing
model (Gemini-2.5 Pro) achieves an accuracy of only 73.7%. Fine-grained
analysis indicates that while existing MLLMs demonstrate strong performance on
tasks where relevant texts are contained within a single or few frames, they
exhibit limited capability in effectively handling tasks that demand holistic
video comprehension. These limitations are especially evident in scenarios that
require spatio-temporal reasoning, cross-frame information integration, or
resistance to language prior bias. Our findings also highlight the importance
of high-resolution visual input and sufficient temporal coverage for reliable
OCR in dynamic video scenarios.
683668aab445f089286ca219
Yang Shi
admin_assigned
2025-05-28T12:19:59.951Z
673c7319d11b1c2e246ead9c
https://cdn-avatars.huggingface.co/v1/production/uploads/673c7319d11b1c2e246ead9c/IjFIO--N7Hm_BOEafhEQv.jpeg
Yang Shi
user
DogNeverSleep
683668aab445f089286ca21a
Huanqian Wang
claimed_verified
2025-05-28T19:14:22.919Z
64ec52887e69909b1ad1ff0b
/avatars/30bb6fda51093fc71e80d0313c13d15e.svg
Lucy Wang
user
Lucywang720
683668aab445f089286ca21b
Wulin Xie
683668aab445f089286ca21c
Huanyao Zhang
683668aab445f089286ca21d
Lijie Zhao
683668aab445f089286ca21e
Yi-Fan Zhang
claimed_verified
2025-07-03T07:42:06.500Z
623d8ca4c29adf5ef6175615
https://cdn-avatars.huggingface.co/v1/production/uploads/623d8ca4c29adf5ef6175615/q7lHao7UPwU1u7YLSP56m.jpeg
Yi-Fan Zhang
user
yifanzhang114
683668aab445f089286ca21f
Xinfeng Li
admin_assigned
2025-05-28T12:20:46.725Z
65af5f8f3db2280ece7fac79
/avatars/66d88b2d744c8d00e11d39a55ab86c2e.svg
Xin-Feng Li
user
xinfeng1i
683668aab445f089286ca220
Chaoyou Fu
683668aab445f089286ca221
Zhuoer Wen
683668aab445f089286ca222
Wenting Liu
683668aab445f089286ca223
Zhuoran Zhang
683668aab445f089286ca224
Xinlong Chen
683668aab445f089286ca225
Bohan Zeng
claimed_verified
2025-06-09T10:13:04.482Z
6671214c92412fd4640714eb
/avatars/48fa84e7bc3bb92ad0192aa26b32de10.svg
bohan zeng
user
zbhpku
683668aab445f089286ca226
Sihan Yang
683668aab445f089286ca227
Yuanxing Zhang
683668aab445f089286ca228
Pengfei Wan
683668aab445f089286ca229
Haotian Wang
683668aab445f089286ca22a
Wenjing Yang
MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in
  Video Scenarios
Multimodal Large Language Models (MLLMs) have achieved considerable accuracy
in Optical Character Recognition (OCR) from static images. However, their
efficacy in video OCR is significantly diminished due to factors such as motion
blur, temporal variations, and visual effects inherent in video content. To
provide clearer guidance for training practical MLLMs, we introduce the
MME-VideoOCR benchmark, which encompasses a comprehensive range of video OCR
application scenarios. MME-VideoOCR features 10 task categories comprising 25
individual tasks and spans 44 diverse scenarios. These tasks extend beyond text
recognition to incorporate deeper comprehension and reasoning of textual
content within videos. The benchmark consists of 1,464 videos with varying
resolutions, aspect ratios, and durations, along with 2,000 meticulously
curated, manually annotated question-answer pairs. We evaluate 18
state-of-the-art MLLMs on MME-VideoOCR, revealing that even the best-performing
model (Gemini-2.5 Pro) achieves an accuracy of only 73.7%. Fine-grained
analysis indicates that while existing MLLMs demonstrate strong performance on
tasks where relevant texts are contained within a single or few frames, they
exhibit limited capability in effectively handling tasks that demand holistic
video comprehension. These limitations are especially evident in scenarios that
require spatio-temporal reasoning, cross-frame information integration, or
resistance to language prior bias. Our findings also highlight the importance
of high-resolution visual input and sufficient temporal coverage for reliable
OCR in dynamic video scenarios.
MLLMs achieve modest accuracy in video OCR due to motion blur, temporal variations, and visual effects; MME-VideoOCR benchmark reveals limitations in spatio-temporal reasoning and language bias.
Multimodal Large Language Models MLLMs Video OCR MME-VideoOCR task categories video comprehension spatio-temporal reasoning cross-frame information integration language prior bias high-resolution visual input temporal coverage dynamic video scenarios