LLM-I: LLMs are Naturally Interleaved Multimodal Creators
We propose LLM-Interleaved (LLM-I), a flexible and dynamic framework that
reframes interleaved image-text generation as a tool-use problem. LLM-I is
designed to overcome the "one-tool" bottleneck of current unified models, which
are limited to synthetic imagery and struggle with tasks requiring factual
grounding or programmatic precision. Our framework empowers a central LLM or
MLLM agent to intelligently orchestrate a diverse toolkit of specialized visual
tools, including online image search, diffusion-based generation, code
execution, and image editing. The agent is trained to select and apply these
tools proficiently via a Reinforcement Learning (RL) framework that features a
hybrid reward system combining rule-based logic with judgments from LLM and
MLLM evaluators. Trained on a diverse new dataset using four different model
backbones, LLM-I demonstrates state-of-the-art performance, outperforming
existing methods by a large margin across four benchmarks. We also introduce a
novel test-time scaling strategy that provides further performance gains.
Project Page: https://github.com/ByteDance-BandAI/LLM-I.
68cb89f75a7803ff3be42e4f
Zirun Guo
claimed_verified
2025-09-18T13:26:06.027Z
67160d7ed0aceb0835d7fc12
/avatars/f4dc2cfffbe5ae2ffcc9410c31759b62.svg
zrguo
user
kkwok
68cb89f75a7803ff3be42e50
Feng Zhang
68cb89f75a7803ff3be42e51
Kai Jia
68cb89f75a7803ff3be42e52
Tao Jin
LLM-I: LLMs are Naturally Interleaved Multimodal Creators
We propose LLM-Interleaved (LLM-I), a flexible and dynamic framework that
reframes interleaved image-text generation as a tool-use problem. LLM-I is
designed to overcome the "one-tool" bottleneck of current unified models, which
are limited to synthetic imagery and struggle with tasks requiring factual
grounding or programmatic precision. Our framework empowers a central LLM or
MLLM agent to intelligently orchestrate a diverse toolkit of specialized visual
tools, including online image search, diffusion-based generation, code
execution, and image editing. The agent is trained to select and apply these
tools proficiently via a Reinforcement Learning (RL) framework that features a
hybrid reward system combining rule-based logic with judgments from LLM and
MLLM evaluators. Trained on a diverse new dataset using four different model
backbones, LLM-I demonstrates state-of-the-art performance, outperforming
existing methods by a large margin across four benchmarks. We also introduce a
novel test-time scaling strategy that provides further performance gains.
Project Page: https://github.com/ByteDance-BandAI/LLM-I.
LLM-Interleaved (LLM-I) is a flexible framework that uses a central LLM to orchestrate a toolkit of specialized visual tools, achieving state-of-the-art performance in image-text generation through reinforcement learning and a novel scaling strategy.
LLM-Interleaved LLM-I interleaved image-text generation tool-use problem unified models synthetic imagery factual grounding programmatic precision diffusion-based generation code execution image editing Reinforcement Learning RL hybrid reward system rule-based logic LLM evaluators MLLM evaluators model backbones state-of-the-art performance benchmarks test-time scaling strategy