Optimal Control Meets Flow Matching: A Principled Route to Multi-Subject
  Fidelity
Text-to-image (T2I) models excel on single-entity prompts but struggle with
multi-subject descriptions, often showing attribute leakage, identity
entanglement, and subject omissions. We introduce the first theoretical
framework with a principled, optimizable objective for steering sampling
dynamics toward multi-subject fidelity. Viewing flow matching (FM) through
stochastic optimal control (SOC), we formulate subject disentanglement as
control over a trained FM sampler. This yields two architecture-agnostic
algorithms: (i) a training-free test-time controller that perturbs the base
velocity with a single-pass update, and (ii) Adjoint Matching, a lightweight
fine-tuning rule that regresses a control network to a backward adjoint signal
while preserving base-model capabilities. The same formulation unifies prior
attention heuristics, extends to diffusion models via a flow-diffusion
correspondence, and provides the first fine-tuning route explicitly designed
for multi-subject fidelity. Empirically, on Stable Diffusion 3.5, FLUX, and
Stable Diffusion XL, both algorithms consistently improve multi-subject
alignment while maintaining base-model style. Test-time control runs
efficiently on commodity GPUs, and fine-tuned controllers trained on limited
prompts generalize to unseen ones. We further highlight FOCUS (Flow Optimal
Control for Unentangled Subjects), which achieves state-of-the-art
multi-subject fidelity across models.
68df48ecdf49fb0df1e03d4c
Eric Tillmann Bill
claimed_verified
2025-10-07T12:28:53.028Z
66fd492153a22cc59893a420
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U12qeuzr4wWNBAJI_vGck.png
Eric
user
ericbill21
68df48ecdf49fb0df1e03d4d
Enis Simsar
claimed_verified
2025-10-03T12:29:01.815Z
63412f2add8853dc7e306a4f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/nqbteeQhS2dPA5LVy7dTD.png
Enis Simsar
user
enisimsar
68df48ecdf49fb0df1e03d4e
Thomas Hofmann
Optimal Control Meets Flow Matching: A Principled Route to Multi-Subject
  Fidelity
Text-to-image (T2I) models excel on single-entity prompts but struggle with
multi-subject descriptions, often showing attribute leakage, identity
entanglement, and subject omissions. We introduce the first theoretical
framework with a principled, optimizable objective for steering sampling
dynamics toward multi-subject fidelity. Viewing flow matching (FM) through
stochastic optimal control (SOC), we formulate subject disentanglement as
control over a trained FM sampler. This yields two architecture-agnostic
algorithms: (i) a training-free test-time controller that perturbs the base
velocity with a single-pass update, and (ii) Adjoint Matching, a lightweight
fine-tuning rule that regresses a control network to a backward adjoint signal
while preserving base-model capabilities. The same formulation unifies prior
attention heuristics, extends to diffusion models via a flow-diffusion
correspondence, and provides the first fine-tuning route explicitly designed
for multi-subject fidelity. Empirically, on Stable Diffusion 3.5, FLUX, and
Stable Diffusion XL, both algorithms consistently improve multi-subject
alignment while maintaining base-model style. Test-time control runs
efficiently on commodity GPUs, and fine-tuned controllers trained on limited
prompts generalize to unseen ones. We further highlight FOCUS (Flow Optimal
Control for Unentangled Subjects), which achieves state-of-the-art
multi-subject fidelity across models.
A theoretical framework and algorithms for improving multi-subject fidelity in text-to-image models through control over sampling dynamics.
flow matching stochastic optimal control subject disentanglement training-free test-time controller Adjoint Matching flow-diffusion correspondence fine-tuning multi-subject alignment Stable Diffusion FLUX Stable Diffusion XL FOCUS