DiarizationLM: Speaker Diarization Post-Processing with Large Language
  Models
In this paper, we introduce DiarizationLM, a framework to leverage large
language models (LLM) to post-process the outputs from a speaker diarization
system. Various goals can be achieved with the proposed framework, such as
improving the readability of the diarized transcript, or reducing the word
diarization error rate (WDER). In this framework, the outputs of the automatic
speech recognition (ASR) and speaker diarization systems are represented as a
compact textual format, which is included in the prompt to an optionally
finetuned LLM. The outputs of the LLM can be used as the refined diarization
results with the desired enhancement. As a post-processing step, this framework
can be easily applied to any off-the-shelf ASR and speaker diarization systems
without retraining existing components. Our experiments show that a finetuned
PaLM 2-S model can reduce the WDER by rel. 25.9% on the Fisher telephone
conversation dataset, and rel. 31% on the Callhome English dataset.
659cc8d41d398a23816921bd
Quan Wang
extracted_confirmed
2024-01-09T13:43:13.822Z
64bea31d81caff7f184ad4a0
https://cdn-avatars.huggingface.co/v1/production/uploads/64bea31d81caff7f184ad4a0/hq4f77BRNoiY92musz9kq.jpeg
Quan
user
wq2012
659cc8d41d398a23816921be
Yiling Huang
659cc8d41d398a23816921bf
Guanlong Zhao
659cc8d41d398a23816921c0
Evan Clark
659cc8d41d398a23816921c1
Wei Xia
659cc8d41d398a23816921c2
Hank Liao
admin_assigned
2024-01-09T09:33:50.308Z
655eeea1a74c042364de33e0
/avatars/3249c16a373becf94bf13155b13d487e.svg
Hank Liao
user
HankLiao
DiarizationLM: Speaker Diarization Post-Processing with Large Language
  Models
In this paper, we introduce DiarizationLM, a framework to leverage large
language models (LLM) to post-process the outputs from a speaker diarization
system. Various goals can be achieved with the proposed framework, such as
improving the readability of the diarized transcript, or reducing the word
diarization error rate (WDER). In this framework, the outputs of the automatic
speech recognition (ASR) and speaker diarization systems are represented as a
compact textual format, which is included in the prompt to an optionally
finetuned LLM. The outputs of the LLM can be used as the refined diarization
results with the desired enhancement. As a post-processing step, this framework
can be easily applied to any off-the-shelf ASR and speaker diarization systems
without retraining existing components. Our experiments show that a finetuned
PaLM 2-S model can reduce the WDER by rel. 25.9% on the Fisher telephone
conversation dataset, and rel. 31% on the Callhome English dataset.
DiarizationLM framework enhances post-processed speaker diarization outputs by utilizing large language models, improving readability and reducing word diarization error rates.
speaker diarization automatic speech recognition large language models LLM finetuned PaLM 2-S WDER Fisher telephone conversation dataset Callhome English dataset