Attention IoU: Examining Biases in CelebA using Attention Maps
Computer vision models have been shown to exhibit and amplify biases across a
wide array of datasets and tasks. Existing methods for quantifying bias in
classification models primarily focus on dataset distribution and model
performance on subgroups, overlooking the internal workings of a model. We
introduce the Attention-IoU (Attention Intersection over Union) metric and
related scores, which use attention maps to reveal biases within a model's
internal representations and identify image features potentially causing the
biases. First, we validate Attention-IoU on the synthetic Waterbirds dataset,
showing that the metric accurately measures model bias. We then analyze the
CelebA dataset, finding that Attention-IoU uncovers correlations beyond
accuracy disparities. Through an investigation of individual attributes through
the protected attribute of Male, we examine the distinct ways biases are
represented in CelebA. Lastly, by subsampling the training set to change
attribute correlations, we demonstrate that Attention-IoU reveals potential
confounding variables not present in dataset labels.
67e4c449672b3d9c9cb8aa4b
Aaron Serianni
claimed_verified
2025-03-27T09:03:27.573Z
67e4c444692ba54248a6b337
/avatars/bd62d0247559d9e9dbfcf87870da251f.svg
Aaron Serianni
user
serianni
67e4c449672b3d9c9cb8aa4c
Tyler Zhu
admin_assigned
2025-03-27T10:16:06.286Z
6301af7259ab5d9dc0a15060
/avatars/0bcda98801877332893a1de169000737.svg
Tyler Zhu
user
tyleryzhu
67e4c449672b3d9c9cb8aa4d
Olga Russakovsky
67e4c449672b3d9c9cb8aa4e
Vikram V. Ramaswamy
Attention IoU: Examining Biases in CelebA using Attention Maps
Computer vision models have been shown to exhibit and amplify biases across a
wide array of datasets and tasks. Existing methods for quantifying bias in
classification models primarily focus on dataset distribution and model
performance on subgroups, overlooking the internal workings of a model. We
introduce the Attention-IoU (Attention Intersection over Union) metric and
related scores, which use attention maps to reveal biases within a model's
internal representations and identify image features potentially causing the
biases. First, we validate Attention-IoU on the synthetic Waterbirds dataset,
showing that the metric accurately measures model bias. We then analyze the
CelebA dataset, finding that Attention-IoU uncovers correlations beyond
accuracy disparities. Through an investigation of individual attributes through
the protected attribute of Male, we examine the distinct ways biases are
represented in CelebA. Lastly, by subsampling the training set to change
attribute correlations, we demonstrate that Attention-IoU reveals potential
confounding variables not present in dataset labels.
The Attention-IoU metric uses attention maps to uncover biases within model's internal representations, identifying image features causing bias and revealing potential confounding variables.
Attention-IoU Attention Intersection over Union attention maps model bias internal representations image features Waterbirds dataset CelebA dataset attribute correlations protected attribute Male confounding variables