Factuality Matters: When Image Generation and Editing Meet Structured
  Visuals
While modern visual generation models excel at creating aesthetically
pleasing natural images, they struggle with producing or editing structured
visuals like charts, diagrams, and mathematical figures, which demand
composition planning, text rendering, and multimodal reasoning for factual
fidelity. To address this, we present the first comprehensive, systematic
investigation of this domain, encompassing data construction, model training,
and an evaluation benchmark. First, we construct a large-scale dataset of 1.3
million high-quality structured image pairs derived from executable drawing
programs and augmented with chain-of-thought reasoning annotations. Building on
it, we train a unified model that integrates a VLM with FLUX.1 Kontext via a
lightweight connector for enhanced multimodal understanding. A three-stage
training curriculum enables progressive feature alignment, knowledge infusion,
and reasoning-augmented generation, further boosted by an external reasoner at
inference time. Finally, we introduce StructBench, a novel benchmark for
generation and editing with over 1,700 challenging instances, and an
accompanying evaluation metric, StructScore, which employs a multi-round Q\&A
protocol to assess fine-grained factual accuracy. Evaluations of 15 models
reveal that even leading closed-source systems remain far from satisfactory.
Our model attains strong editing performance, and inference-time reasoning
yields consistent gains across diverse architectures. By releasing the dataset,
model, and benchmark, we aim to advance unified multimodal foundations for
structured visuals.
68e47ef4e4e093a7044e4d27
Le Zhuo
68e47ef4e4e093a7044e4d28
Songhao Han
claimed_verified
2025-10-07T12:27:05.438Z
662885b9b87483ae5a9ee5c9
https://cdn-avatars.huggingface.co/v1/production/uploads/662885b9b87483ae5a9ee5c9/fLzCrWizo_hy7zGuAqFwk.jpeg
Songhao Han
user
hshjerry0315
68e47ef4e4e093a7044e4d29
Yuandong Pu
claimed_verified
2025-10-13T10:10:28.139Z
625d5b9f0bec31f086e04cd9
https://cdn-avatars.huggingface.co/v1/production/uploads/1650285458447-noauth.jpeg
YuandongPu
user
Andrew613
68e47ef4e4e093a7044e4d2a
Boxiang Qiu
claimed_verified
2025-10-09T12:49:14.045Z
675721d18ebab4432daaf579
https://cdn-avatars.huggingface.co/v1/production/uploads/675721d18ebab4432daaf579/WKN6BB2Hx80UnxdpdnPtP.png
qbx
user
qiukingballball
68e47ef4e4e093a7044e4d2b
Sayak Paul
claimed_verified
2025-10-07T12:27:03.087Z
5f7fbd813e94f16a85448745
https://cdn-avatars.huggingface.co/v1/production/uploads/1649681653581-5f7fbd813e94f16a85448745.jpeg
Sayak Paul
user
sayakpaul
68e47ef4e4e093a7044e4d2c
Yue Liao
68e47ef4e4e093a7044e4d2d
Yihao Liu
68e47ef4e4e093a7044e4d2e
Jie Shao
68e47ef4e4e093a7044e4d2f
Xi Chen
68e47ef4e4e093a7044e4d30
Si Liu
68e47ef4e4e093a7044e4d31
Hongsheng Li
Factuality Matters: When Image Generation and Editing Meet Structured
  Visuals
While modern visual generation models excel at creating aesthetically
pleasing natural images, they struggle with producing or editing structured
visuals like charts, diagrams, and mathematical figures, which demand
composition planning, text rendering, and multimodal reasoning for factual
fidelity. To address this, we present the first comprehensive, systematic
investigation of this domain, encompassing data construction, model training,
and an evaluation benchmark. First, we construct a large-scale dataset of 1.3
million high-quality structured image pairs derived from executable drawing
programs and augmented with chain-of-thought reasoning annotations. Building on
it, we train a unified model that integrates a VLM with FLUX.1 Kontext via a
lightweight connector for enhanced multimodal understanding. A three-stage
training curriculum enables progressive feature alignment, knowledge infusion,
and reasoning-augmented generation, further boosted by an external reasoner at
inference time. Finally, we introduce StructBench, a novel benchmark for
generation and editing with over 1,700 challenging instances, and an
accompanying evaluation metric, StructScore, which employs a multi-round Q\&A
protocol to assess fine-grained factual accuracy. Evaluations of 15 models
reveal that even leading closed-source systems remain far from satisfactory.
Our model attains strong editing performance, and inference-time reasoning
yields consistent gains across diverse architectures. By releasing the dataset,
model, and benchmark, we aim to advance unified multimodal foundations for
structured visuals.
A comprehensive investigation into generating and editing structured visuals using a unified model integrating a VLM with FLUX Kontext, achieving strong performance and introducing a new benchmark and evaluation metric.
VLM FLUX Kontext chain-of-thought reasoning multimodal understanding three-stage training curriculum feature alignment knowledge infusion reasoning-augmented generation StructBench StructScore multi-round Q&A protocol