Unlearning Comparator: A Visual Analytics System for Comparative
  Evaluation of Machine Unlearning Methods
Machine Unlearning (MU) aims to remove target training data from a trained
model so that the removed data no longer influences the model's behavior,
fulfilling "right to be forgotten" obligations under data privacy laws. Yet, we
observe that researchers in this rapidly emerging field face challenges in
analyzing and understanding the behavior of different MU methods, especially in
terms of three fundamental principles in MU: accuracy, efficiency, and privacy.
Consequently, they often rely on aggregate metrics and ad-hoc evaluations,
making it difficult to accurately assess the trade-offs between methods. To
fill this gap, we introduce a visual analytics system, Unlearning Comparator,
designed to facilitate the systematic evaluation of MU methods. Our system
supports two important tasks in the evaluation process: model comparison and
attack simulation. First, it allows the user to compare the behaviors of two
models, such as a model generated by a certain method and a retrained baseline,
at class-, instance-, and layer-levels to better understand the changes made
after unlearning. Second, our system simulates membership inference attacks
(MIAs) to evaluate the privacy of a method, where an attacker attempts to
determine whether specific data samples were part of the original training set.
We evaluate our system through a case study visually analyzing prominent MU
methods and demonstrate that it helps the user not only understand model
behaviors but also gain insights that can inform the improvement of MU methods.
68a3f629b65388761d07453b
Jaeung Lee
claimed_verified
2025-08-19T08:04:15.966Z
6663459bd6bf635504ec7dfc
/avatars/8727370401076a42666f8d4a05cf463d.svg
jaeunglee
user
jaeunglee
68a3f629b65388761d07453c
Suhyeon Yu
68a3f629b65388761d07453d
Yurim Jang
68a3f629b65388761d07453e
Simon S. Woo
68a3f629b65388761d07453f
Jaemin Jo
Unlearning Comparator: A Visual Analytics System for Comparative
  Evaluation of Machine Unlearning Methods
Machine Unlearning (MU) aims to remove target training data from a trained
model so that the removed data no longer influences the model's behavior,
fulfilling "right to be forgotten" obligations under data privacy laws. Yet, we
observe that researchers in this rapidly emerging field face challenges in
analyzing and understanding the behavior of different MU methods, especially in
terms of three fundamental principles in MU: accuracy, efficiency, and privacy.
Consequently, they often rely on aggregate metrics and ad-hoc evaluations,
making it difficult to accurately assess the trade-offs between methods. To
fill this gap, we introduce a visual analytics system, Unlearning Comparator,
designed to facilitate the systematic evaluation of MU methods. Our system
supports two important tasks in the evaluation process: model comparison and
attack simulation. First, it allows the user to compare the behaviors of two
models, such as a model generated by a certain method and a retrained baseline,
at class-, instance-, and layer-levels to better understand the changes made
after unlearning. Second, our system simulates membership inference attacks
(MIAs) to evaluate the privacy of a method, where an attacker attempts to
determine whether specific data samples were part of the original training set.
We evaluate our system through a case study visually analyzing prominent MU
methods and demonstrate that it helps the user not only understand model
behaviors but also gain insights that can inform the improvement of MU methods.
A visual analytics system, Unlearning Comparator, facilitates the evaluation of Machine Unlearning methods by comparing model behaviors and simulating membership inference attacks to assess privacy.
Machine Unlearning MU right to be forgotten data privacy laws accuracy efficiency privacy visual analytics system model comparison attack simulation membership inference attacks MIAs