Federated Computation of ROC and PR Curves
Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are
fundamental tools for evaluating machine learning classifiers, offering
detailed insights into the trade-offs between true positive rate vs. false
positive rate (ROC) or precision vs. recall (PR). However, in Federated
Learning (FL) scenarios, where data is distributed across multiple clients,
computing these curves is challenging due to privacy and communication
constraints. Specifically, the server cannot access raw prediction scores and
class labels, which are used to compute the ROC and PR curves in a centralized
setting. In this paper, we propose a novel method for approximating ROC and PR
curves in a federated setting by estimating quantiles of the prediction score
distribution under distributed differential privacy. We provide theoretical
bounds on the Area Error (AE) between the true and estimated curves,
demonstrating the trade-offs between approximation accuracy, privacy, and
communication cost. Empirical results on real-world datasets demonstrate that
our method achieves high approximation accuracy with minimal communication and
strong privacy guarantees, making it practical for privacy-preserving model
evaluation in federated systems.
68e4cfd6e4e093a7044e4e93
Xuefeng Xu
claimed_verified
2025-10-07T12:25:29.207Z
65d836973381a5be278a475a
https://cdn-avatars.huggingface.co/v1/production/uploads/65d836973381a5be278a475a/dgOfY5qZbuUY6HK6s1WsK.jpeg
Xuefeng Xu
user
xuefeng-xu
68e4cfd6e4e093a7044e4e94
Graham Cormode
Federated Computation of ROC and PR Curves
Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are
fundamental tools for evaluating machine learning classifiers, offering
detailed insights into the trade-offs between true positive rate vs. false
positive rate (ROC) or precision vs. recall (PR). However, in Federated
Learning (FL) scenarios, where data is distributed across multiple clients,
computing these curves is challenging due to privacy and communication
constraints. Specifically, the server cannot access raw prediction scores and
class labels, which are used to compute the ROC and PR curves in a centralized
setting. In this paper, we propose a novel method for approximating ROC and PR
curves in a federated setting by estimating quantiles of the prediction score
distribution under distributed differential privacy. We provide theoretical
bounds on the Area Error (AE) between the true and estimated curves,
demonstrating the trade-offs between approximation accuracy, privacy, and
communication cost. Empirical results on real-world datasets demonstrate that
our method achieves high approximation accuracy with minimal communication and
strong privacy guarantees, making it practical for privacy-preserving model
evaluation in federated systems.
A method for approximating ROC and PR curves in federated learning under distributed differential privacy ensures high accuracy, minimal communication, and strong privacy guarantees.
Receiver Operating Characteristic (ROC) Precision-Recall (PR) curves Federated Learning (FL) distributed differential privacy quantiles prediction score distribution Area Error (AE)