Stay on topic with Classifier-Free Guidance
Classifier-Free Guidance (CFG) has recently emerged in text-to-image
generation as a lightweight technique to encourage prompt-adherence in
generations. In this work, we demonstrate that CFG can be used broadly as an
inference-time technique in pure language modeling. We show that CFG (1)
improves the performance of Pythia, GPT-2 and LLaMA-family models across an
array of tasks: Q\&A, reasoning, code generation, and machine translation,
achieving SOTA on LAMBADA with LLaMA-7B over PaLM-540B; (2) brings improvements
equivalent to a model with twice the parameter-count; (3) can stack alongside
other inference-time methods like Chain-of-Thought and Self-Consistency,
yielding further improvements in difficult tasks; (4) can be used to increase
the faithfulness and coherence of assistants in challenging form-driven and
content-driven prompts: in a human evaluation we show a 75\% preference for
GPT4All using CFG over baseline.
64a222628856c528ea9be2fa
Guillaume Sanchez
extracted_confirmed
2023-07-03T11:45:46.176Z
6318b35869ef5467e18f0277
/avatars/1f89cd6a94e4a1e5fea6767a78ba5a0c.svg
Vermeille
user
Vermifuge
64a222628856c528ea9be2fb
Honglu Fan
claimed_verified
2023-07-03T11:47:18.848Z
635bdaf7dc371b8f91029184
/avatars/7e3769763c0e3f3ff83f3020aef60768.svg
Honglu
user
quintic
64a222628856c528ea9be2fc
Alexander Spangher
extracted_confirmed
2024-12-02T18:29:31.416Z
63b67943029518c6bbfa4af5
/avatars/9ac6db8b8541162ee625a8929a2736ee.svg
Alexander Spangher
user
alex2awesome
64a222628856c528ea9be2fd
Elad Levi
extracted_confirmed
2023-07-03T05:01:41.486Z
635d618494e5b275ca73b844
/avatars/8cdaac6591a12b252612b99094e00959.svg
Levi
user
Eladlev
64a222628856c528ea9be2fe
Pawan Sasanka Ammanamanchi
extracted_pending
2023-07-03T01:20:34.881Z
5f04beab4ec31d33a72116d4
https://cdn-avatars.huggingface.co/v1/production/uploads/1594146435878-noauth.jpeg
Pawan Sasanka Ammanamanchi
user
paws
64a222628856c528ea9be2ff
Stella Biderman
extracted_confirmed
2024-02-13T05:23:37.865Z
60347d3660e3dd96631c9093
https://cdn-avatars.huggingface.co/v1/production/uploads/60347d3660e3dd96631c9093/B3fuZer5N04tZIAYrLnz4.jpeg
Stella Biderman
user
stellaathena
Stay on topic with Classifier-Free Guidance
Classifier-Free Guidance (CFG) has recently emerged in text-to-image
generation as a lightweight technique to encourage prompt-adherence in
generations. In this work, we demonstrate that CFG can be used broadly as an
inference-time technique in pure language modeling. We show that CFG (1)
improves the performance of Pythia, GPT-2 and LLaMA-family models across an
array of tasks: Q\&A, reasoning, code generation, and machine translation,
achieving SOTA on LAMBADA with LLaMA-7B over PaLM-540B; (2) brings improvements
equivalent to a model with twice the parameter-count; (3) can stack alongside
other inference-time methods like Chain-of-Thought and Self-Consistency,
yielding further improvements in difficult tasks; (4) can be used to increase
the faithfulness and coherence of assistants in challenging form-driven and
content-driven prompts: in a human evaluation we show a 75\% preference for
GPT4All using CFG over baseline.
Classifier-Free Guidance enhances performance across various language modeling tasks and improves the faithfulness and coherence of AI assistants, outperforming models with higher parameter counts.
Classifier-Free Guidance Pythia GPT-2 LLaMA-family models Q&A reasoning code generation machine translation LAMBADA PaLM-540B Chain-of-Thought Self-Consistency human evaluation