Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on
  Materials Characterization
Materials characterization is fundamental to acquiring materials information,
revealing the processing-microstructure-property relationships that guide
material design and optimization. While multimodal large language models
(MLLMs) have recently shown promise in generative and predictive tasks within
materials science, their capacity to understand real-world characterization
imaging data remains underexplored. To bridge this gap, we present MatCha, the
first benchmark for materials characterization image understanding, comprising
1,500 questions that demand expert-level domain expertise. MatCha encompasses
four key stages of materials research comprising 21 distinct tasks, each
designed to reflect authentic challenges faced by materials scientists. Our
evaluation of state-of-the-art MLLMs on MatCha reveals a significant
performance gap compared to human experts. These models exhibit degradation
when addressing questions requiring higher-level expertise and sophisticated
visual perception. Simple few-shot and chain-of-thought prompting struggle to
alleviate these limitations. These findings highlight that existing MLLMs still
exhibit limited adaptability to real-world materials characterization
scenarios. We hope MatCha will facilitate future research in areas such as new
material discovery and autonomous scientific agents. MatCha is available at
https://github.com/FreedomIntelligence/MatCha.
68cd48333df9ac65e93dc7d0
Zhengzhao Lai
claimed_verified
2025-09-19T13:09:46.842Z
679f64c6d196d603a98fa831
/avatars/f9d16a2461862bafe8f8509503fb748e.svg
Zhengzhao Lai
user
onlyairnopods
68cd48333df9ac65e93dc7d1
Youbin Zheng
68cd48333df9ac65e93dc7d2
Zhenyang Cai
claimed_verified
2025-09-21T13:12:57.390Z
64f1a34f2c5c8b767916447e
https://cdn-avatars.huggingface.co/v1/production/uploads/64f1a34f2c5c8b767916447e/uak2CsMAnxW8q4dwyAOBN.jpeg
Zhenyang Cai
user
Eric3200
68cd48333df9ac65e93dc7d3
Haonan Lyu
68cd48333df9ac65e93dc7d4
Jinpu Yang
68cd48333df9ac65e93dc7d5
Hongqing Liang
68cd48333df9ac65e93dc7d6
Yan Hu
68cd48333df9ac65e93dc7d7
Benyou Wang
Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on
  Materials Characterization
Materials characterization is fundamental to acquiring materials information,
revealing the processing-microstructure-property relationships that guide
material design and optimization. While multimodal large language models
(MLLMs) have recently shown promise in generative and predictive tasks within
materials science, their capacity to understand real-world characterization
imaging data remains underexplored. To bridge this gap, we present MatCha, the
first benchmark for materials characterization image understanding, comprising
1,500 questions that demand expert-level domain expertise. MatCha encompasses
four key stages of materials research comprising 21 distinct tasks, each
designed to reflect authentic challenges faced by materials scientists. Our
evaluation of state-of-the-art MLLMs on MatCha reveals a significant
performance gap compared to human experts. These models exhibit degradation
when addressing questions requiring higher-level expertise and sophisticated
visual perception. Simple few-shot and chain-of-thought prompting struggle to
alleviate these limitations. These findings highlight that existing MLLMs still
exhibit limited adaptability to real-world materials characterization
scenarios. We hope MatCha will facilitate future research in areas such as new
material discovery and autonomous scientific agents. MatCha is available at
https://github.com/FreedomIntelligence/MatCha.
MatCha is a benchmark for evaluating the performance of multimodal large language models in understanding materials characterization images, revealing significant limitations compared to human experts.
multimodal large language models MLLMs benchmark materials characterization image understanding few-shot prompting chain-of-thought prompting autonomous scientific agents