Adapting General Disentanglement-Based Speaker Anonymization for
  Enhanced Emotion Preservation
A general disentanglement-based speaker anonymization system typically
separates speech into content, speaker, and prosody features using individual
encoders. This paper explores how to adapt such a system when a new speech
attribute, for example, emotion, needs to be preserved to a greater extent.
While existing systems are good at anonymizing speaker embeddings, they are not
designed to preserve emotion. Two strategies for this are examined. First, we
show that integrating emotion embeddings from a pre-trained emotion encoder can
help preserve emotional cues, even though this approach slightly compromises
privacy protection. Alternatively, we propose an emotion compensation strategy
as a post-processing step applied to anonymized speaker embeddings. This
conceals the original speaker's identity and reintroduces the emotional traits
lost during speaker embedding anonymization. Specifically, we model the emotion
attribute using support vector machines to learn separate boundaries for each
emotion. During inference, the original speaker embedding is processed in two
ways: one, by an emotion indicator to predict emotion and select the
emotion-matched SVM accurately; and two, by a speaker anonymizer to conceal
speaker characteristics. The anonymized speaker embedding is then modified
along the corresponding SVM boundary towards an enhanced emotional direction to
save the emotional cues. The proposed strategies are also expected to be useful
for adapting a general disentanglement-based speaker anonymization system to
preserve other target paralinguistic attributes, with potential for a range of
downstream tasks.
66bc88c7780d735f1754c00b
Xiaoxiao Miao
66bc88c7780d735f1754c00c
Yuxiang Zhang
66bc88c7780d735f1754c00d
Xin Wang
66bc88c7780d735f1754c00e
Natalia Tomashenko
admin_assigned
2024-08-14T11:11:14.807Z
619d132b0facfda0c4ebc4e2
https://cdn-avatars.huggingface.co/v1/production/uploads/1637684705612-619d132b0facfda0c4ebc4e2.jpeg
Natalia Tomashenko
user
Natalia-T
66bc88c7780d735f1754c00f
Donny Cheng Lock Soh
66bc88c7780d735f1754c010
Ian Mcloughlin
Adapting General Disentanglement-Based Speaker Anonymization for
  Enhanced Emotion Preservation
A general disentanglement-based speaker anonymization system typically
separates speech into content, speaker, and prosody features using individual
encoders. This paper explores how to adapt such a system when a new speech
attribute, for example, emotion, needs to be preserved to a greater extent.
While existing systems are good at anonymizing speaker embeddings, they are not
designed to preserve emotion. Two strategies for this are examined. First, we
show that integrating emotion embeddings from a pre-trained emotion encoder can
help preserve emotional cues, even though this approach slightly compromises
privacy protection. Alternatively, we propose an emotion compensation strategy
as a post-processing step applied to anonymized speaker embeddings. This
conceals the original speaker's identity and reintroduces the emotional traits
lost during speaker embedding anonymization. Specifically, we model the emotion
attribute using support vector machines to learn separate boundaries for each
emotion. During inference, the original speaker embedding is processed in two
ways: one, by an emotion indicator to predict emotion and select the
emotion-matched SVM accurately; and two, by a speaker anonymizer to conceal
speaker characteristics. The anonymized speaker embedding is then modified
along the corresponding SVM boundary towards an enhanced emotional direction to
save the emotional cues. The proposed strategies are also expected to be useful
for adapting a general disentanglement-based speaker anonymization system to
preserve other target paralinguistic attributes, with potential for a range of
downstream tasks.
A speaker anonymization system is adapted to preserve emotional cues while maintaining speaker privacy using integrated emotion embeddings or a post-processing emotion compensation strategy.
disentanglement-based speaker anonymization speaker embeddings emotion embeddings pre-trained emotion encoder support vector machines SVM emotion indicator