Temporal Alignment Guidance: On-Manifold Sampling in Diffusion Models
Diffusion models have achieved remarkable success as generative models.
However, even a well-trained model can accumulate errors throughout the
generation process. These errors become particularly problematic when arbitrary
guidance is applied to steer samples toward desired properties, which often
breaks sample fidelity. In this paper, we propose a general solution to address
the off-manifold phenomenon observed in diffusion models. Our approach
leverages a time predictor to estimate deviations from the desired data
manifold at each timestep, identifying that a larger time gap is associated
with reduced generation quality. We then design a novel guidance mechanism,
`Temporal Alignment Guidance' (TAG), attracting the samples back to the desired
manifold at every timestep during generation. Through extensive experiments, we
demonstrate that TAG consistently produces samples closely aligned with the
desired manifold at each timestep, leading to significant improvements in
generation quality across various downstream tasks.
68ef3928486b78128f0e356b
Youngrok Park
68ef3928486b78128f0e356c
Hojung Jung
68ef3928486b78128f0e356d
Sangmin Bae
68ef3928486b78128f0e356e
Se-Young Yun
Temporal Alignment Guidance: On-Manifold Sampling in Diffusion Models
Diffusion models have achieved remarkable success as generative models.
However, even a well-trained model can accumulate errors throughout the
generation process. These errors become particularly problematic when arbitrary
guidance is applied to steer samples toward desired properties, which often
breaks sample fidelity. In this paper, we propose a general solution to address
the off-manifold phenomenon observed in diffusion models. Our approach
leverages a time predictor to estimate deviations from the desired data
manifold at each timestep, identifying that a larger time gap is associated
with reduced generation quality. We then design a novel guidance mechanism,
`Temporal Alignment Guidance' (TAG), attracting the samples back to the desired
manifold at every timestep during generation. Through extensive experiments, we
demonstrate that TAG consistently produces samples closely aligned with the
desired manifold at each timestep, leading to significant improvements in
generation quality across various downstream tasks.