SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for
  Enterprise Use
Enterprise customers are increasingly adopting Large Language Models (LLMs)
for critical communication tasks, such as drafting emails, crafting sales
pitches, and composing casual messages. Deploying such models across different
regions requires them to understand diverse cultural and linguistic contexts
and generate safe and respectful responses. For enterprise applications, it is
crucial to mitigate reputational risks, maintain trust, and ensure compliance
by effectively identifying and handling unsafe or offensive language. To
address this, we introduce SweEval, a benchmark simulating real-world scenarios
with variations in tone (positive or negative) and context (formal or
informal). The prompts explicitly instruct the model to include specific swear
words while completing the task. This benchmark evaluates whether LLMs comply
with or resist such inappropriate instructions and assesses their alignment
with ethical frameworks, cultural nuances, and language comprehension
capabilities. In order to advance research in building ethically aligned AI
systems for enterprise use and beyond, we release the dataset and code:
https://github.com/amitbcp/multilingual_profanity.
68349ace56ca97d1b2f1eb7c
Hitesh Laxmichand Patel
68349ace56ca97d1b2f1eb7d
Amit Agarwal
claimed_verified
2025-06-02T07:49:14.382Z
6321a50a6b1992383fa00db1
/avatars/92311b4a5e8bcc8f5477747621334fc8.svg
Amit Agarwal
user
amitbcp
68349ace56ca97d1b2f1eb7e
Arion Das
claimed_verified
2025-05-27T07:53:29.445Z
655c848048320febfc45db74
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/-An-36AfvExD_EfLFXHOP.jpeg
Arion Das
user
ariondas
68349ace56ca97d1b2f1eb7f
Bhargava Kumar
68349ace56ca97d1b2f1eb80
Srikant Panda
68349ace56ca97d1b2f1eb81
Priyaranjan Pattnayak
68349ace56ca97d1b2f1eb82
Taki Hasan Rafi
68349ace56ca97d1b2f1eb83
Tejaswini Kumar
68349ace56ca97d1b2f1eb84
Dong-Kyu Chae
SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for
  Enterprise Use
Enterprise customers are increasingly adopting Large Language Models (LLMs)
for critical communication tasks, such as drafting emails, crafting sales
pitches, and composing casual messages. Deploying such models across different
regions requires them to understand diverse cultural and linguistic contexts
and generate safe and respectful responses. For enterprise applications, it is
crucial to mitigate reputational risks, maintain trust, and ensure compliance
by effectively identifying and handling unsafe or offensive language. To
address this, we introduce SweEval, a benchmark simulating real-world scenarios
with variations in tone (positive or negative) and context (formal or
informal). The prompts explicitly instruct the model to include specific swear
words while completing the task. This benchmark evaluates whether LLMs comply
with or resist such inappropriate instructions and assesses their alignment
with ethical frameworks, cultural nuances, and language comprehension
capabilities. In order to advance research in building ethically aligned AI
systems for enterprise use and beyond, we release the dataset and code:
https://github.com/amitbcp/multilingual_profanity.
SweEval is a benchmark for evaluating Large Language Models' compliance with ethical guidelines and cultural nuances when instructed to include offensive language.
Large Language Models LLMs SweEval benchmark ethical alignment cultural nuances language comprehension ethical frameworks