Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of
  Experts
While score-based generative models are the model of choice across diverse
domains, there are limited tools available for controlling inference-time
behavior in a principled manner, e.g. for composing multiple pretrained models.
Existing classifier-free guidance methods use a simple heuristic to mix
conditional and unconditional scores to approximately sample from conditional
distributions. However, such methods do not approximate the intermediate
distributions, necessitating additional 'corrector' steps. In this work, we
provide an efficient and principled method for sampling from a sequence of
annealed, geometric-averaged, or product distributions derived from pretrained
score-based models. We derive a weighted simulation scheme which we call
Feynman-Kac Correctors (FKCs) based on the celebrated Feynman-Kac formula by
carefully accounting for terms in the appropriate partial differential
equations (PDEs). To simulate these PDEs, we propose Sequential Monte Carlo
(SMC) resampling algorithms that leverage inference-time scaling to improve
sampling quality. We empirically demonstrate the utility of our methods by
proposing amortized sampling via inference-time temperature annealing,
improving multi-objective molecule generation using pretrained models, and
improving classifier-free guidance for text-to-image generation. Our code is
available at https://github.com/martaskrt/fkc-diffusion.
67d0b23d33256c62f20dbc1f
Marta Skreta
claimed_verified
2025-03-12T08:41:41.189Z
65a49ef27ec6af0f956a5c61
https://cdn-avatars.huggingface.co/v1/production/uploads/65a49ef27ec6af0f956a5c61/OGJ1OVezooF7dUn1SiEKG.jpeg
Marta Skreta
user
mskrt
67d0b23d33256c62f20dbc20
Tara Akhound-Sadegh
67d0b23d33256c62f20dbc21
Viktor Ohanesian
67d0b23d33256c62f20dbc22
Roberto Bondesan
67d0b23d33256c62f20dbc23
Alán Aspuru-Guzik
67d0b23d33256c62f20dbc24
Arnaud Doucet
67d0b23d33256c62f20dbc25
Rob Brekelmans
67d0b23d33256c62f20dbc26
Alexander Tong
67d0b23d33256c62f20dbc27
Kirill Neklyudov
Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of
  Experts
While score-based generative models are the model of choice across diverse
domains, there are limited tools available for controlling inference-time
behavior in a principled manner, e.g. for composing multiple pretrained models.
Existing classifier-free guidance methods use a simple heuristic to mix
conditional and unconditional scores to approximately sample from conditional
distributions. However, such methods do not approximate the intermediate
distributions, necessitating additional 'corrector' steps. In this work, we
provide an efficient and principled method for sampling from a sequence of
annealed, geometric-averaged, or product distributions derived from pretrained
score-based models. We derive a weighted simulation scheme which we call
Feynman-Kac Correctors (FKCs) based on the celebrated Feynman-Kac formula by
carefully accounting for terms in the appropriate partial differential
equations (PDEs). To simulate these PDEs, we propose Sequential Monte Carlo
(SMC) resampling algorithms that leverage inference-time scaling to improve
sampling quality. We empirically demonstrate the utility of our methods by
proposing amortized sampling via inference-time temperature annealing,
improving multi-objective molecule generation using pretrained models, and
improving classifier-free guidance for text-to-image generation. Our code is
available at https://github.com/martaskrt/fkc-diffusion.
The study introduces an efficient and principled method for sampling from sequences of distributions derived from pretrained score-based models using Feynman-Kac Correctors and Sequential Monte Carlo, enhancing tasks like molecule generation and text-to-image generation.
score-based generative models classifier-free guidance Feynman-Kac Correctors Sequential Monte Carlo (SMC) inference-time temperature annealing multi-objective molecule generation text-to-image generation