Faithful Persona-based Conversational Dataset Generation with Large
  Language Models
High-quality conversational datasets are essential for developing AI models
that can communicate with users. One way to foster deeper interactions between
a chatbot and its user is through personas, aspects of the user's character
that provide insights into their personality, motivations, and behaviors.
Training Natural Language Processing (NLP) models on a diverse and
comprehensive persona-based dataset can lead to conversational models that
create a deeper connection with the user, and maintain their engagement. In
this paper, we leverage the power of Large Language Models (LLMs) to create a
large, high-quality conversational dataset from a seed dataset. We propose a
Generator-Critic architecture framework to expand the initial dataset, while
improving the quality of its conversations. The Generator is an LLM prompted to
output conversations. The Critic consists of a mixture of expert LLMs that
control the quality of the generated conversations. These experts select the
best generated conversations, which we then use to improve the Generator. We
release Synthetic-Persona-Chat, consisting of 20k conversations seeded from
Persona-Chat. We evaluate the quality of Synthetic-Persona-Chat and our
generation framework on different dimensions through extensive experiments, and
observe that the losing rate of Synthetic-Persona-Chat against Persona-Chat
during Turing test decreases from 17.2% to 8.8% over three iterations.
657fb590138b7e3914bdda20
Pegah Jandaghi
extracted_confirmed
2023-12-18T07:07:37.039Z
648cc6cb3945927fe05e2c30
/avatars/75ace8f1e9ce516fc4543e6c11bbe6e7.svg
Pegah
user
JPegah
657fb590138b7e3914bdda21
XiangHai Sheng
admin_assigned
2023-12-18T11:29:53.750Z
6430960c2a7fbdfb5258249f
/avatars/20f95ed65cc321f03f2c510ff1ab9b61.svg
Xianghai Sheng
user
nkstnkst
657fb590138b7e3914bdda22
Xinyi Bai
admin_assigned
2023-12-18T11:30:01.435Z
6438baec27a3c24cf4a58d01
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/mRebsJ0VluWWoGhuN6jUb.jpeg
Xinyi Bai
user
wwwbxy123
657fb590138b7e3914bdda23
Jay Pujara
657fb590138b7e3914bdda24
Hakim Sidahmed
Faithful Persona-based Conversational Dataset Generation with Large
  Language Models
High-quality conversational datasets are essential for developing AI models
that can communicate with users. One way to foster deeper interactions between
a chatbot and its user is through personas, aspects of the user's character
that provide insights into their personality, motivations, and behaviors.
Training Natural Language Processing (NLP) models on a diverse and
comprehensive persona-based dataset can lead to conversational models that
create a deeper connection with the user, and maintain their engagement. In
this paper, we leverage the power of Large Language Models (LLMs) to create a
large, high-quality conversational dataset from a seed dataset. We propose a
Generator-Critic architecture framework to expand the initial dataset, while
improving the quality of its conversations. The Generator is an LLM prompted to
output conversations. The Critic consists of a mixture of expert LLMs that
control the quality of the generated conversations. These experts select the
best generated conversations, which we then use to improve the Generator. We
release Synthetic-Persona-Chat, consisting of 20k conversations seeded from
Persona-Chat. We evaluate the quality of Synthetic-Persona-Chat and our
generation framework on different dimensions through extensive experiments, and
observe that the losing rate of Synthetic-Persona-Chat against Persona-Chat
during Turing test decreases from 17.2% to 8.8% over three iterations.
A Generator-Critic architecture using Large Language Models expands a seed dataset to create a high-quality conversational dataset, improving engagement in AI conversations.
Large Language Models (LLMs) Generator-Critic architecture Persona-Chat Synthetic-Persona-Chat Turing test