AutoMat: Enabling Automated Crystal Structure Reconstruction from
  Microscopy via Agentic Tool Use
Machine learning-based interatomic potentials and force fields depend
critically on accurate atomic structures, yet such data are scarce due to the
limited availability of experimentally resolved crystals. Although
atomic-resolution electron microscopy offers a potential source of structural
data, converting these images into simulation-ready formats remains
labor-intensive and error-prone, creating a bottleneck for model training and
validation. We introduce AutoMat, an end-to-end, agent-assisted pipeline that
automatically transforms scanning transmission electron microscopy (STEM)
images into atomic crystal structures and predicts their physical properties.
AutoMat combines pattern-adaptive denoising, physics-guided template retrieval,
symmetry-aware atomic reconstruction, fast relaxation and property prediction
via MatterSim, and coordinated orchestration across all stages. We propose the
first dedicated STEM2Mat-Bench for this task and evaluate performance using
lattice RMSD, formation energy MAE, and structure-matching success rate. By
orchestrating external tool calls, AutoMat enables a text-only LLM to
outperform vision-language models in this domain, achieving closed-loop
reasoning throughout the pipeline. In large-scale experiments over 450
structure samples, AutoMat substantially outperforms existing multimodal large
language models and tools. These results validate both AutoMat and
STEM2Mat-Bench, marking a key step toward bridging microscopy and atomistic
simulation in materials science.The code and dataset are publicly available at
https://github.com/yyt-2378/AutoMat and
https://huggingface.co/datasets/yaotianvector/STEM2Mat.
682c057b36dd2dbca3931150
Yaotian Yang
claimed_verified
2025-05-22T07:18:15.239Z
64ca1b79ec9a33183aa3bd29
/avatars/cafeafeccced927aac986575338a804c.svg
yang
user
yaotianvector
682c057b36dd2dbca3931151
Yiwen Tang
claimed_verified
2025-05-22T07:18:12.714Z
6552f1ad5d55ccb20e9142a0
/avatars/0e3e80cba64b5ae0bc5638694ac33dbf.svg
Ivan Tang
user
IvanTang
682c057b36dd2dbca3931152
Yizhe Chen
682c057b36dd2dbca3931153
Xiao Chen
682c057b36dd2dbca3931154
Jiangjie Qiu
682c057b36dd2dbca3931155
Hao Xiong
682c057b36dd2dbca3931156
Haoyu Yin
682c057b36dd2dbca3931157
Zhiyao Luo
682c057b36dd2dbca3931158
Yifei Zhang
682c057b36dd2dbca3931159
Sijia Tao
682c057b36dd2dbca393115a
Wentao Li
682c057b36dd2dbca393115b
Qinghua Zhang
682c057b36dd2dbca393115c
Yuqiang Li
682c057b36dd2dbca393115d
Wanli Ouyang
682c057b36dd2dbca393115e
Bin Zhao
682c057b36dd2dbca393115f
Xiaonan Wang
682c057b36dd2dbca3931160
Fei Wei
AutoMat: Enabling Automated Crystal Structure Reconstruction from
  Microscopy via Agentic Tool Use
Machine learning-based interatomic potentials and force fields depend
critically on accurate atomic structures, yet such data are scarce due to the
limited availability of experimentally resolved crystals. Although
atomic-resolution electron microscopy offers a potential source of structural
data, converting these images into simulation-ready formats remains
labor-intensive and error-prone, creating a bottleneck for model training and
validation. We introduce AutoMat, an end-to-end, agent-assisted pipeline that
automatically transforms scanning transmission electron microscopy (STEM)
images into atomic crystal structures and predicts their physical properties.
AutoMat combines pattern-adaptive denoising, physics-guided template retrieval,
symmetry-aware atomic reconstruction, fast relaxation and property prediction
via MatterSim, and coordinated orchestration across all stages. We propose the
first dedicated STEM2Mat-Bench for this task and evaluate performance using
lattice RMSD, formation energy MAE, and structure-matching success rate. By
orchestrating external tool calls, AutoMat enables a text-only LLM to
outperform vision-language models in this domain, achieving closed-loop
reasoning throughout the pipeline. In large-scale experiments over 450
structure samples, AutoMat substantially outperforms existing multimodal large
language models and tools. These results validate both AutoMat and
STEM2Mat-Bench, marking a key step toward bridging microscopy and atomistic
simulation in materials science.The code and dataset are publicly available at
https://github.com/yyt-2378/AutoMat and
https://huggingface.co/datasets/yaotianvector/STEM2Mat.
AutoMat, an agent-assisted pipeline, transforms atomic-resolution STEM images into simulation-ready atomic crystal structures and predicts their properties, overcoming the bottleneck in data availability and processing.
end-to-end pipeline agent-assisted scanning transmission electron microscopy (STEM) pattern-adaptive denoising physics-guided template retrieval symmetry-aware atomic reconstruction fast relaxation property prediction MatterSim STEM2Mat-Bench lattice RMSD formation energy MAE structure-matching success rate multimodal large language models