From Sparse to Dense: GPT-4 Summarization with Chain of Density
  Prompting
Selecting the ``right'' amount of information to include in a summary is a
difficult task. A good summary should be detailed and entity-centric without
being overly dense and hard to follow. To better understand this tradeoff, we
solicit increasingly dense GPT-4 summaries with what we refer to as a ``Chain
of Density'' (CoD) prompt. Specifically, GPT-4 generates an initial
entity-sparse summary before iteratively incorporating missing salient entities
without increasing the length. Summaries generated by CoD are more abstractive,
exhibit more fusion, and have less of a lead bias than GPT-4 summaries
generated by a vanilla prompt. We conduct a human preference study on 100 CNN
DailyMail articles and find that that humans prefer GPT-4 summaries that are
more dense than those generated by a vanilla prompt and almost as dense as
human written summaries. Qualitative analysis supports the notion that there
exists a tradeoff between informativeness and readability. 500 annotated CoD
summaries, as well as an extra 5,000 unannotated summaries, are freely
available on HuggingFace
(https://huggingface.co/datasets/griffin/chain_of_density).
64fe6bc848411fc789d33df8
Griffin Adams
extracted_confirmed
2023-09-11T01:23:17.772Z
611693254ef9fdfbf45dc4d6
https://cdn-avatars.huggingface.co/v1/production/uploads/1628869402886-noauth.jpeg
Griffin Adams
user
griffin
64fe6bc848411fc789d33df9
Alexander Fabbri
claimed_verified
2023-09-18T17:14:41.062Z
626427701ed8d81e47ab6899
/avatars/7c89167744703771e3f163665cdb66cc.svg
Alexander Fabbri
user
alexfabbri
64fe6bc848411fc789d33dfa
Faisal Ladhak
admin_assigned
2023-09-11T11:03:39.182Z
61ee4f4f53fdc44f4e5e197a
/avatars/f23f56809805c3e06c580d7420f48494.svg
Faisal Ladhak
user
fladhak
64fe6bc848411fc789d33dfb
Eric Lehman
extracted_pending
2023-09-11T01:22:17.555Z
63be486fd7dea2e13e6133a0
/avatars/79a7b799044ea48a11899284777eb22e.svg
Eric Lehman
user
lehmer16
64fe6bc848411fc789d33dfc
Noémie Elhadad
From Sparse to Dense: GPT-4 Summarization with Chain of Density
  Prompting
Selecting the ``right'' amount of information to include in a summary is a
difficult task. A good summary should be detailed and entity-centric without
being overly dense and hard to follow. To better understand this tradeoff, we
solicit increasingly dense GPT-4 summaries with what we refer to as a ``Chain
of Density'' (CoD) prompt. Specifically, GPT-4 generates an initial
entity-sparse summary before iteratively incorporating missing salient entities
without increasing the length. Summaries generated by CoD are more abstractive,
exhibit more fusion, and have less of a lead bias than GPT-4 summaries
generated by a vanilla prompt. We conduct a human preference study on 100 CNN
DailyMail articles and find that that humans prefer GPT-4 summaries that are
more dense than those generated by a vanilla prompt and almost as dense as
human written summaries. Qualitative analysis supports the notion that there
exists a tradeoff between informativeness and readability. 500 annotated CoD
summaries, as well as an extra 5,000 unannotated summaries, are freely
available on HuggingFace
(https://huggingface.co/datasets/griffin/chain_of_density).
CoD prompt in GPT-4 improves summary density, informativeness, and human preference over vanilla prompts, with tradeoffs between readability and detail.
GPT-4 Chain of Density (CoD) entity-sparse salient entities human preference study vanilla prompt informativeness readability HuggingFace