Privacy Reasoning in Ambiguous Contexts
Ren Yi, Octavian Suciu, Adrian Gascon, Sarah Meiklejohn, Eugene Bagdasarian, Marco Gruteser
We study the ability of language models to reason about appropriate information disclosure--a central aspect of the evolving field of agent privacy. While previous works have focused on evaluating the models' ability to align with human decisions, we examine the role of ambiguity and missing context on model accuracy, e.g., precision and recall, when rendering information sharing decisions. We identify context ambiguity as a crucial barrier for high performance in privacy assessments. By designing a framework for context disambiguation, we show that model-generated decision rationales can reveal ambiguities and that systematically disambiguating context based on these rationales leads to significant accuracy improvements (up to 13.3\% in precision and up to 22.3\% in recall) as well as reductions in prompt sensitivity. Overall, our results indicate that approaches for context disambiguation are a promising way forward to enhance agent privacy reasoning.