ChatGPT-steered Editing Instructor for Customization of Abstractive
  Summarization
Tailoring outputs of large language models, such as ChatGPT, to specific user
needs remains a challenge despite their impressive generation quality. In this
paper, we propose a tri-agent generation pipeline consisting of a generator, an
instructor, and an editor to enhance the customization of generated outputs.
The generator produces an initial output, the user-specific instructor
generates editing instructions, and the editor generates a revised output
aligned with user preferences. The inference-only large language model
(ChatGPT) serves as both the generator and the editor, while a smaller model
acts as the user-specific instructor to guide the generation process toward
user needs. The instructor is trained using editor-steered reinforcement
learning, leveraging feedback from the large-scale editor model to optimize
instruction generation. Experimental results on two abstractive summarization
datasets demonstrate the effectiveness of our approach in generating outputs
that better fulfill user expectations.
645454d0cd09ceba0e1596b7
Wen Xiao
645454d0cd09ceba0e1596b8
Yujia Xie
645454d0cd09ceba0e1596b9
Giuseppe Carenini
645454d0cd09ceba0e1596ba
Pengcheng He
admin_assigned
2023-06-02T13:39:54.233Z
5efd09cf49ed724c8a135868
/avatars/af12bc94657979677a9f26183f0c9727.svg
Pengcheng He
user
DeBERTa
ChatGPT-steered Editing Instructor for Customization of Abstractive
  Summarization
Tailoring outputs of large language models, such as ChatGPT, to specific user
needs remains a challenge despite their impressive generation quality. In this
paper, we propose a tri-agent generation pipeline consisting of a generator, an
instructor, and an editor to enhance the customization of generated outputs.
The generator produces an initial output, the user-specific instructor
generates editing instructions, and the editor generates a revised output
aligned with user preferences. The inference-only large language model
(ChatGPT) serves as both the generator and the editor, while a smaller model
acts as the user-specific instructor to guide the generation process toward
user needs. The instructor is trained using editor-steered reinforcement
learning, leveraging feedback from the large-scale editor model to optimize
instruction generation. Experimental results on two abstractive summarization
datasets demonstrate the effectiveness of our approach in generating outputs
that better fulfill user expectations.
A tri-agent pipeline for enhancing customizability of large language model outputs includes a generator, an instructor, and an editor, with the instructor being optimized through reinforcement learning guided by the editor.
large language models ChatGPT tri-agent generation pipeline generator instructor editor reinforcement learning editor-steered reinforcement learning abstractive summarization