BiasGym: Fantastic Biases and How to Find (and Remove) Them
Understanding biases and stereotypes encoded in the weights of Large Language
Models (LLMs) is crucial for developing effective mitigation strategies. Biased
behaviour is often subtle and non-trivial to isolate, even when deliberately
elicited, making systematic analysis and debiasing particularly challenging. To
address this, we introduce BiasGym, a simple, cost-effective, and generalizable
framework for reliably injecting, analyzing, and mitigating conceptual
associations within LLMs. BiasGym consists of two components: BiasInject, which
injects specific biases into the model via token-based fine-tuning while
keeping the model frozen, and BiasScope, which leverages these injected signals
to identify and steer the components responsible for biased behavior. Our
method enables consistent bias elicitation for mechanistic analysis, supports
targeted debiasing without degrading performance on downstream tasks, and
generalizes to biases unseen during training. We demonstrate the effectiveness
of BiasGym in reducing real-world stereotypes (e.g., people from a country
being `reckless drivers') and in probing fictional associations (e.g., people
from a country having `blue skin'), showing its utility for both safety
interventions and interpretability research.
689c224ffab6fdd2e52ac8aa
Sekh Mainul Islam
claimed_verified
2025-08-13T07:10:36.205Z
637390430938c0754238276c
https://cdn-avatars.huggingface.co/v1/production/uploads/637390430938c0754238276c/O7IIJdZsS9yjznh780XwA.jpeg
Sekh Mainul Islam
user
sekhcopenlu
689c224ffab6fdd2e52ac8ab
Nadav Borenstein
689c224ffab6fdd2e52ac8ac
Siddhesh Milind Pawar
689c224ffab6fdd2e52ac8ad
Haeun Yu
689c224ffab6fdd2e52ac8ae
Arnav Arora
689c224ffab6fdd2e52ac8af
Isabelle Augenstein
BiasGym: Fantastic Biases and How to Find (and Remove) Them
Understanding biases and stereotypes encoded in the weights of Large Language
Models (LLMs) is crucial for developing effective mitigation strategies. Biased
behaviour is often subtle and non-trivial to isolate, even when deliberately
elicited, making systematic analysis and debiasing particularly challenging. To
address this, we introduce BiasGym, a simple, cost-effective, and generalizable
framework for reliably injecting, analyzing, and mitigating conceptual
associations within LLMs. BiasGym consists of two components: BiasInject, which
injects specific biases into the model via token-based fine-tuning while
keeping the model frozen, and BiasScope, which leverages these injected signals
to identify and steer the components responsible for biased behavior. Our
method enables consistent bias elicitation for mechanistic analysis, supports
targeted debiasing without degrading performance on downstream tasks, and
generalizes to biases unseen during training. We demonstrate the effectiveness
of BiasGym in reducing real-world stereotypes (e.g., people from a country
being `reckless drivers') and in probing fictional associations (e.g., people
from a country having `blue skin'), showing its utility for both safety
interventions and interpretability research.
BiasGym is a framework for injecting, analyzing, and mitigating biases in Large Language Models through token-based fine-tuning and signal analysis.
Large Language Models BiasGym BiasInject BiasScope token-based fine-tuning conceptual associations bias elicitation mechanistic analysis targeted debiasing real-world stereotypes fictional associations