Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence
  Embeddings for Automatic Dialog Flow Extraction
Efficiently deriving structured workflows from unannotated dialogs remains an
underexplored and formidable challenge in computational linguistics. Automating
this process could significantly accelerate the manual design of workflows in
new domains and enable the grounding of large language models in
domain-specific flowcharts, enhancing transparency and controllability. In this
paper, we introduce Dialog2Flow (D2F) embeddings, which differ from
conventional sentence embeddings by mapping utterances to a latent space where
they are grouped according to their communicative and informative functions
(i.e., the actions they represent). D2F allows for modeling dialogs as
continuous trajectories in a latent space with distinct action-related regions.
By clustering D2F embeddings, the latent space is quantized, and dialogs can be
converted into sequences of region/action IDs, facilitating the extraction of
the underlying workflow. To pre-train D2F, we build a comprehensive dataset by
unifying twenty task-oriented dialog datasets with normalized per-turn action
annotations. We also introduce a novel soft contrastive loss that leverages the
semantic information of these actions to guide the representation learning
process, showing superior performance compared to standard supervised
contrastive loss. Evaluation against various sentence embeddings, including
dialog-specific ones, demonstrates that D2F yields superior qualitative and
quantitative results across diverse domains.
671b671a25260fa066e1a4da
Sergio Burdisso
extracted_confirmed
2024-10-25T09:38:42.902Z
61d626d9e3a234cfcb617dca
https://cdn-avatars.huggingface.co/v1/production/uploads/61d626d9e3a234cfcb617dca/kJD3oxckDgiaEnYwGth6p.jpeg
Sergio Burdisso
user
sergioburdisso
671b671a25260fa066e1a4db
Srikanth Madikeri
671b671a25260fa066e1a4dc
Petr Motlicek
Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence
  Embeddings for Automatic Dialog Flow Extraction
Efficiently deriving structured workflows from unannotated dialogs remains an
underexplored and formidable challenge in computational linguistics. Automating
this process could significantly accelerate the manual design of workflows in
new domains and enable the grounding of large language models in
domain-specific flowcharts, enhancing transparency and controllability. In this
paper, we introduce Dialog2Flow (D2F) embeddings, which differ from
conventional sentence embeddings by mapping utterances to a latent space where
they are grouped according to their communicative and informative functions
(i.e., the actions they represent). D2F allows for modeling dialogs as
continuous trajectories in a latent space with distinct action-related regions.
By clustering D2F embeddings, the latent space is quantized, and dialogs can be
converted into sequences of region/action IDs, facilitating the extraction of
the underlying workflow. To pre-train D2F, we build a comprehensive dataset by
unifying twenty task-oriented dialog datasets with normalized per-turn action
annotations. We also introduce a novel soft contrastive loss that leverages the
semantic information of these actions to guide the representation learning
process, showing superior performance compared to standard supervised
contrastive loss. Evaluation against various sentence embeddings, including
dialog-specific ones, demonstrates that D2F yields superior qualitative and
quantitative results across diverse domains.
Dialog2Flow embeddings facilitate the automated extraction of workflows from unannotated dialogs by mapping utterances to a latent space and leveraging a novel soft contrastive loss for representation learning.
Dialog2Flow (D2F) embeddings latent space communicative and informative functions continuous trajectories action-related regions clustering region/action IDs soft contrastive loss semantic information representation learning task-oriented dialog datasets action annotations sentence embeddings qualitative and quantitative results