PLANNER: Generating Diversified Paragraph via Latent Language Diffusion
  Model
Autoregressive models for text sometimes generate repetitive and low-quality
output because errors accumulate during the steps of generation. This issue is
often attributed to exposure bias - the difference between how a model is
trained, and how it is used during inference. Denoising diffusion models
provide an alternative approach in which a model can revisit and revise its
output. However, they can be computationally expensive and prior efforts on
text have led to models that produce less fluent output compared to
autoregressive models, especially for longer text and paragraphs. In this
paper, we propose PLANNER, a model that combines latent semantic diffusion with
autoregressive generation, to generate fluent text while exercising global
control over paragraphs. The model achieves this by combining an autoregressive
"decoding" module with a "planning" module that uses latent diffusion to
generate semantic paragraph embeddings in a coarse-to-fine manner. The proposed
method is evaluated on various conditional generation tasks, and results on
semantic generation, text completion and summarization show its effectiveness
in generating high-quality long-form text in an efficient manner.
647eb35d1a1fcad2fdc6006c
Yizhe Zhang
claimed_verified
2023-06-06T17:23:37.181Z
640d1d188aee167ccda1a52a
/avatars/c320b53a31ffba41bd9ad71da99ebc83.svg
Yizhe Zhang
user
YizheZ
647eb35d1a1fcad2fdc6006d
Jiatao Gu
claimed_verified
2023-06-06T21:03:22.734Z
6164e72d73996c363c52e66d
https://cdn-avatars.huggingface.co/v1/production/uploads/1634002684894-noauth.png
Jiatao Gu
user
thomagram
647eb35d1a1fcad2fdc6006e
Zhuofeng Wu
claimed_verified
2024-02-26T21:10:06.449Z
63c75671656e7822e23df874
/avatars/6fbe669287405b2b47cafcf4961cfa75.svg
Zhuofeng Wu
user
cserxy
647eb35d1a1fcad2fdc6006f
Shuangfei Zhai
claimed_verified
2024-12-13T15:25:19.804Z
675bf04240726aac79ae8e7b
/avatars/8ae95255dfce91a6f5e76f14eae9e9e5.svg
Shuangfei Zhai
user
shuangfei
647eb35d1a1fcad2fdc60070
Josh Susskind
claimed_verified
2023-06-06T17:23:41.773Z
6470d2247fd7ecdbd0ec3cc9
/avatars/52c5eca12499a1aa9bd49c43d4f20685.svg
Joshua M. Susskind
user
jsusskind
647eb35d1a1fcad2fdc60071
Navdeep Jaitly
PLANNER: Generating Diversified Paragraph via Latent Language Diffusion
  Model
Autoregressive models for text sometimes generate repetitive and low-quality
output because errors accumulate during the steps of generation. This issue is
often attributed to exposure bias - the difference between how a model is
trained, and how it is used during inference. Denoising diffusion models
provide an alternative approach in which a model can revisit and revise its
output. However, they can be computationally expensive and prior efforts on
text have led to models that produce less fluent output compared to
autoregressive models, especially for longer text and paragraphs. In this
paper, we propose PLANNER, a model that combines latent semantic diffusion with
autoregressive generation, to generate fluent text while exercising global
control over paragraphs. The model achieves this by combining an autoregressive
"decoding" module with a "planning" module that uses latent diffusion to
generate semantic paragraph embeddings in a coarse-to-fine manner. The proposed
method is evaluated on various conditional generation tasks, and results on
semantic generation, text completion and summarization show its effectiveness
in generating high-quality long-form text in an efficient manner.
PLANNER effectively generates high-quality long-form text by combining autoregressive generation with latent semantic diffusion for improved fluency and coherence.
denoising diffusion models latent semantic diffusion autoregressive models exposure bias semantic generation text completion summarization high-quality long-form text