Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering
Fighting online hate speech is a challenge that is usually addressed using
Natural Language Processing via automatic detection and removal of hate
content. Besides this approach, counter narratives have emerged as an effective
tool employed by NGOs to respond to online hate on social media platforms. For
this reason, Natural Language Generation is currently being studied as a way to
automatize counter narrative writing. However, the existing resources necessary
to train NLG models are limited to 2-turn interactions (a hate speech and a
counter narrative as response), while in real life, interactions can consist of
multiple turns. In this paper, we present a hybrid approach for dialogical data
collection, which combines the intervention of human expert annotators over
machine generated dialogues obtained using 19 different configurations. The
result of this work is DIALOCONAN, the first dataset comprising over 3000
fictitious multi-turn dialogues between a hater and an NGO operator, covering 6
targets of hate.