From Charts to Code: A Hierarchical Benchmark for Multimodal Models
We introduce Chart2Code, a new benchmark for evaluating the chart
understanding and code generation capabilities of large multimodal models
(LMMs). Chart2Code is explicitly designed from a user-driven perspective,
capturing diverse real-world scenarios and progressively increasing task
difficulty. It consists of three levels: Level 1 (Chart Reproduction)
reproduces charts from a reference figure and user query; Level 2 (Chart
Editing) involves complex modifications such as changing chart types or adding
elements; and Level 3 (Long-Table to Chart Generation) requires models to
transform long, information-dense tables into faithful charts following user
instructions. To our knowledge, this is the first hierarchical benchmark that
reflects practical chart2code usage while systematically scaling task
complexity. In total, Chart2Code contains 2,023 tasks across 22 chart types,
paired with multi-level evaluation metrics that assess both code correctness
and the visual fidelity of rendered charts. We benchmark 25 state-of-the-art
(SoTA) LMMs, including both proprietary and the latest open-source models such
as GPT-5, Qwen2.5-VL, InternVL3/3.5, MiMo-VL, and Seed-1.6-VL. Experimental
results demonstrate that even the SoTA model GPT-5 averages only 0.57 on
code-based evaluation and 0.22 on chart-quality assessment across the editing
tasks, underscoring the difficulty of Chart2Code. We anticipate this benchmark
will drive advances in multimodal reasoning and foster the development of more
robust and general-purpose LMMs. Our code and data are available on Chart2Code.
68f867b07669bcaeecce0e2d
Jiahao Tang
68f867b07669bcaeecce0e2e
Henry Hengyuan Zhao
68f867b07669bcaeecce0e2f
Lijian Wu
68f867b07669bcaeecce0e30
Yifei Tao
68f867b07669bcaeecce0e31
Dongxing Mao
68f867b07669bcaeecce0e32
Yang Wan
68f867b07669bcaeecce0e33
Jingru Tan
68f867b07669bcaeecce0e34
Min Zeng
68f867b07669bcaeecce0e35
Min Li
68f867b07669bcaeecce0e36
Alex Jinpeng Wang
From Charts to Code: A Hierarchical Benchmark for Multimodal Models
We introduce Chart2Code, a new benchmark for evaluating the chart
understanding and code generation capabilities of large multimodal models
(LMMs). Chart2Code is explicitly designed from a user-driven perspective,
capturing diverse real-world scenarios and progressively increasing task
difficulty. It consists of three levels: Level 1 (Chart Reproduction)
reproduces charts from a reference figure and user query; Level 2 (Chart
Editing) involves complex modifications such as changing chart types or adding
elements; and Level 3 (Long-Table to Chart Generation) requires models to
transform long, information-dense tables into faithful charts following user
instructions. To our knowledge, this is the first hierarchical benchmark that
reflects practical chart2code usage while systematically scaling task
complexity. In total, Chart2Code contains 2,023 tasks across 22 chart types,
paired with multi-level evaluation metrics that assess both code correctness
and the visual fidelity of rendered charts. We benchmark 25 state-of-the-art
(SoTA) LMMs, including both proprietary and the latest open-source models such
as GPT-5, Qwen2.5-VL, InternVL3/3.5, MiMo-VL, and Seed-1.6-VL. Experimental
results demonstrate that even the SoTA model GPT-5 averages only 0.57 on
code-based evaluation and 0.22 on chart-quality assessment across the editing
tasks, underscoring the difficulty of Chart2Code. We anticipate this benchmark
will drive advances in multimodal reasoning and foster the development of more
robust and general-purpose LMMs. Our code and data are available on Chart2Code.
Chart2Code is a hierarchical benchmark for evaluating the chart understanding and code generation capabilities of large multimodal models, featuring three levels of increasing complexity and diverse real-world scenarios.
multimodal models LMMs Chart2Code Chart Reproduction Chart Editing Long-Table to Chart Generation hierarchical benchmark code correctness visual fidelity GPT-5 Qwen2.5-VL InternVL3/3.5 MiMo-VL Seed-1.6-VL