Symbolic Graphics Programming with Large Language Models
Large language models (LLMs) excel at program synthesis, yet their ability to
produce symbolic graphics programs (SGPs) that render into precise visual
content remains underexplored. We study symbolic graphics programming, where
the goal is to generate an SGP from a natural-language description. This task
also serves as a lens into how LLMs understand the visual world by prompting
them to generate images rendered from SGPs. Among various SGPs, our paper
sticks to scalable vector graphics (SVGs). We begin by examining the extent to
which LLMs can generate SGPs. To this end, we introduce SGP-GenBench, a
comprehensive benchmark covering object fidelity, scene fidelity, and
compositionality (attribute binding, spatial relations, numeracy). On
SGP-GenBench, we discover that frontier proprietary models substantially
outperform open-source models, and performance correlates well with general
coding capabilities. Motivated by this gap, we aim to improve LLMs' ability to
generate SGPs. We propose a reinforcement learning (RL) with verifiable rewards
approach, where a format-validity gate ensures renderable SVG, and a
cross-modal reward aligns text and the rendered image via strong vision
encoders (e.g., SigLIP for text-image and DINO for image-image). Applied to
Qwen-2.5-7B, our method substantially improves SVG generation quality and
semantics, achieving performance on par with frontier systems. We further
analyze training dynamics, showing that RL induces (i) finer decomposition of
objects into controllable primitives and (ii) contextual details that improve
scene coherence. Our results demonstrate that symbolic graphics programming
offers a precise and interpretable lens on cross-modal grounding.
68be49a7c123124955ef5fd0
Yamei Chen
68be49a7c123124955ef5fd1
Haoquan Zhang
claimed_verified
2025-09-09T13:49:45.528Z
67ff7f687351095d4b606b84
https://cdn-avatars.huggingface.co/v1/production/uploads/67ff7f687351095d4b606b84/KhNPmbBC3zghuP5h1MK-c.png
Haoquan Zhang
user
haoquan03
68be49a7c123124955ef5fd2
Yangyi Huang
claimed_verified
2025-09-08T07:01:53.437Z
630461923926de1f7ec7a93a
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/GoUU2qRfLB2wlcciwVBrz.png
Yangyi, Huang
user
YangyiH
68be49a7c123124955ef5fd3
Zeju Qiu
68be49a7c123124955ef5fd4
Kaipeng Zhang
68be49a7c123124955ef5fd5
Yandong Wen
68be49a7c123124955ef5fd6
Weiyang Liu
claimed_verified
2025-09-12T16:20:50.395Z
648905d1a15c43c791d4381f
https://cdn-avatars.huggingface.co/v1/production/uploads/648905d1a15c43c791d4381f/GpqGBzsLiMHX0gWZEz3qn.jpeg
Weiyang Liu
user
wy1iu
Symbolic Graphics Programming with Large Language Models
Large language models (LLMs) excel at program synthesis, yet their ability to
produce symbolic graphics programs (SGPs) that render into precise visual
content remains underexplored. We study symbolic graphics programming, where
the goal is to generate an SGP from a natural-language description. This task
also serves as a lens into how LLMs understand the visual world by prompting
them to generate images rendered from SGPs. Among various SGPs, our paper
sticks to scalable vector graphics (SVGs). We begin by examining the extent to
which LLMs can generate SGPs. To this end, we introduce SGP-GenBench, a
comprehensive benchmark covering object fidelity, scene fidelity, and
compositionality (attribute binding, spatial relations, numeracy). On
SGP-GenBench, we discover that frontier proprietary models substantially
outperform open-source models, and performance correlates well with general
coding capabilities. Motivated by this gap, we aim to improve LLMs' ability to
generate SGPs. We propose a reinforcement learning (RL) with verifiable rewards
approach, where a format-validity gate ensures renderable SVG, and a
cross-modal reward aligns text and the rendered image via strong vision
encoders (e.g., SigLIP for text-image and DINO for image-image). Applied to
Qwen-2.5-7B, our method substantially improves SVG generation quality and
semantics, achieving performance on par with frontier systems. We further
analyze training dynamics, showing that RL induces (i) finer decomposition of
objects into controllable primitives and (ii) contextual details that improve
scene coherence. Our results demonstrate that symbolic graphics programming
offers a precise and interpretable lens on cross-modal grounding.
LLMs generate SVGs from natural-language descriptions using a reinforcement learning approach with verifiable rewards, improving performance and scene coherence.
symbolic graphics programs SGPs scalable vector graphics SVGs SGP-GenBench reinforcement learning RL format-validity gate cross-modal reward SigLIP DINO Qwen-2.5-7B cross-modal grounding