Hatevolution: What Static Benchmarks Don't Tell Us
Language changes over time, including in the hate speech domain, which
evolves quickly following social dynamics and cultural shifts. While NLP
research has investigated the impact of language evolution on model training
and has proposed several solutions for it, its impact on model benchmarking
remains under-explored. Yet, hate speech benchmarks play a crucial role to
ensure model safety. In this paper, we empirically evaluate the robustness of
20 language models across two evolving hate speech experiments, and we show the
temporal misalignment between static and time-sensitive evaluations. Our
findings call for time-sensitive linguistic benchmarks in order to correctly
and reliably evaluate language models in the hate speech domain.
68513a7d8a68fee7f6ba4c2e
Chiara Di Bonaventura
claimed_verified
2025-06-17T13:06:24.238Z
62de70fb86220b5cb895e199
https://cdn-avatars.huggingface.co/v1/production/uploads/62de70fb86220b5cb895e199/k7w8pTBIvCwkrTksQJ7GM.jpeg
Chiara Di Bonaventura
user
dibo
68513a7d8a68fee7f6ba4c2f
Barbara McGillivray
68513a7d8a68fee7f6ba4c30
Yulan He
68513a7d8a68fee7f6ba4c31
Albert Meroño-Peñuela
Hatevolution: What Static Benchmarks Don't Tell Us
Language changes over time, including in the hate speech domain, which
evolves quickly following social dynamics and cultural shifts. While NLP
research has investigated the impact of language evolution on model training
and has proposed several solutions for it, its impact on model benchmarking
remains under-explored. Yet, hate speech benchmarks play a crucial role to
ensure model safety. In this paper, we empirically evaluate the robustness of
20 language models across two evolving hate speech experiments, and we show the
temporal misalignment between static and time-sensitive evaluations. Our
findings call for time-sensitive linguistic benchmarks in order to correctly
and reliably evaluate language models in the hate speech domain.
Empirical evaluation reveals temporal misalignment in the robustness of language models on evolving hate speech benchmarks, highlighting the need for time-sensitive linguistic assessments.
NLP language models hate speech benchmarks temporal misalignment time-sensitive evaluations language evolution