Rendering-Aware Reinforcement Learning for Vector Graphics Generation
Scalable Vector Graphics (SVG) offer a powerful format for representing
visual designs as interpretable code. Recent advances in vision-language models
(VLMs) have enabled high-quality SVG generation by framing the problem as a
code generation task and leveraging large-scale pretraining. VLMs are
particularly suitable for this task as they capture both global semantics and
fine-grained visual patterns, while transferring knowledge across vision,
natural language, and code domains. However, existing VLM approaches often
struggle to produce faithful and efficient SVGs because they never observe the
rendered images during training. Although differentiable rendering for
autoregressive SVG code generation remains unavailable, rendered outputs can
still be compared to original inputs, enabling evaluative feedback suitable for
reinforcement learning (RL). We introduce RLRF(Reinforcement Learning from
Rendering Feedback), an RL method that enhances SVG generation in
autoregressive VLMs by leveraging feedback from rendered SVG outputs. Given an
input image, the model generates SVG roll-outs that are rendered and compared
to the original image to compute a reward. This visual fidelity feedback guides
the model toward producing more accurate, efficient, and semantically coherent
SVGs. RLRF significantly outperforms supervised fine-tuning, addressing common
failure modes and enabling precise, high-quality SVG generation with strong
structural understanding and generalization.
68372ea2ad0fc935fff6bd1e
Juan A. Rodriguez
claimed_verified
2025-05-29T07:43:41.138Z
63a614d264f470027818b066
https://cdn-avatars.huggingface.co/v1/production/uploads/63a614d264f470027818b066/Q5Eih2VqD4NaHB_MkNp32.jpeg
Juan A. Rodriguez
user
joanrodai
68372ea2ad0fc935fff6bd1f
Haotian Zhang
claimed_verified
2025-05-30T06:54:54.945Z
63163a2d95b55e2621d23e4d
https://cdn-avatars.huggingface.co/v1/production/uploads/63163a2d95b55e2621d23e4d/WZokE6kUIwvV8ztB2d0u9.png
Haotian Zhang
user
hz2475
68372ea2ad0fc935fff6bd20
Abhay Puri
68372ea2ad0fc935fff6bd21
Aarash Feizi
68372ea2ad0fc935fff6bd22
Rishav Pramanik
claimed_verified
2025-05-29T07:43:43.396Z
62bb84f82ada492aa5775709
https://cdn-avatars.huggingface.co/v1/production/uploads/62bb84f82ada492aa5775709/jv4yKL75t8QzHDHLbhBPT.png
Rishav Pramanik
user
rishavpramanik
68372ea2ad0fc935fff6bd23
Pascal Wichmann
68372ea2ad0fc935fff6bd24
Arnab Mondal
68372ea2ad0fc935fff6bd25
Mohammad Reza Samsami
68372ea2ad0fc935fff6bd26
Rabiul Awal
claimed_verified
2025-05-28T20:10:16.612Z
60edf5e03203a5daf7d3912e
https://cdn-avatars.huggingface.co/v1/production/uploads/60edf5e03203a5daf7d3912e/fq1AzGutKI_qojhevc03P.jpeg
Rabiul Awal
user
rabiulawal
68372ea2ad0fc935fff6bd27
Perouz Taslakian
68372ea2ad0fc935fff6bd28
Spandana Gella
claimed_verified
2025-06-11T16:02:31.535Z
65ca8745d64d82c92fa7c71f
https://cdn-avatars.huggingface.co/v1/production/uploads/65ca8745d64d82c92fa7c71f/xayDYjs_LoAsa8BEgDlhb.jpeg
G
user
spandanagella
68372ea2ad0fc935fff6bd29
Sai Rajeswar
68372ea2ad0fc935fff6bd2a
David Vazquez
68372ea2ad0fc935fff6bd2b
Christopher Pal
68372ea2ad0fc935fff6bd2c
Marco Pedersoli
Rendering-Aware Reinforcement Learning for Vector Graphics Generation
Scalable Vector Graphics (SVG) offer a powerful format for representing
visual designs as interpretable code. Recent advances in vision-language models
(VLMs) have enabled high-quality SVG generation by framing the problem as a
code generation task and leveraging large-scale pretraining. VLMs are
particularly suitable for this task as they capture both global semantics and
fine-grained visual patterns, while transferring knowledge across vision,
natural language, and code domains. However, existing VLM approaches often
struggle to produce faithful and efficient SVGs because they never observe the
rendered images during training. Although differentiable rendering for
autoregressive SVG code generation remains unavailable, rendered outputs can
still be compared to original inputs, enabling evaluative feedback suitable for
reinforcement learning (RL). We introduce RLRF(Reinforcement Learning from
Rendering Feedback), an RL method that enhances SVG generation in
autoregressive VLMs by leveraging feedback from rendered SVG outputs. Given an
input image, the model generates SVG roll-outs that are rendered and compared
to the original image to compute a reward. This visual fidelity feedback guides
the model toward producing more accurate, efficient, and semantically coherent
SVGs. RLRF significantly outperforms supervised fine-tuning, addressing common
failure modes and enabling precise, high-quality SVG generation with strong
structural understanding and generalization.
RLRF, a reinforcement learning method utilizing rendering feedback, enhances SVG generation in VLMs, improving accuracy and efficiency by comparing rendered SVGs to original images.
vision-language models code generation large-scale pretraining global semantics fine-grained visual patterns differentiable rendering reinforcement learning autoregressive VLMs SVG roll-outs visual fidelity feedback supervised fine-tuning structural understanding generalization