Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness
Computer-Use Agents (CUAs) are an increasingly deployed class of agents that
take actions on GUIs to accomplish user goals. In this paper, we show that CUAs
consistently exhibit Blind Goal-Directedness (BGD): a bias to pursue goals
regardless of feasibility, safety, reliability, or context. We characterize
three prevalent patterns of BGD: (i) lack of contextual reasoning, (ii)
assumptions and decisions under ambiguity, and (iii) contradictory or
infeasible goals. We develop BLIND-ACT, a benchmark of 90 tasks capturing these
three patterns. Built on OSWorld, BLIND-ACT provides realistic environments and
employs LLM-based judges to evaluate agent behavior, achieving 93.75% agreement
with human annotations. We use BLIND-ACT to evaluate nine frontier models,
including Claude Sonnet and Opus 4, Computer-Use-Preview, and GPT-5, observing
high average BGD rates (80.8%) across them. We show that BGD exposes subtle
risks that arise even when inputs are not directly harmful. While
prompting-based interventions lower BGD levels, substantial risk persists,
highlighting the need for stronger training- or inference-time interventions.
Qualitative analysis reveals observed failure modes: execution-first bias
(focusing on how to act over whether to act), thought-action disconnect
(execution diverging from reasoning), and request-primacy (justifying actions
due to user request). Identifying BGD and introducing BLIND-ACT establishes a
foundation for future research on studying and mitigating this fundamental risk
and ensuring safe CUA deployment.
68df30a8df49fb0df1e03c43
Erfan Shayegani
claimed_verified
2025-10-05T12:47:18.195Z
64a0784d7b57fab3a5d63868
https://cdn-avatars.huggingface.co/v1/production/uploads/64a0784d7b57fab3a5d63868/iqA0qF0nYAstSWsSS94RH.png
Erfan Shayegani 😈
user
Erfan-Shayegani
68df30a8df49fb0df1e03c44
Keegan Hines
68df30a8df49fb0df1e03c45
Yue Dong
68df30a8df49fb0df1e03c46
Nael Abu-Ghazaleh
68df30a8df49fb0df1e03c47
Roman Lutz
68df30a8df49fb0df1e03c48
Spencer Whitehead
68df30a8df49fb0df1e03c49
Vidhisha Balachandran
68df30a8df49fb0df1e03c4a
Besmira Nushi
68df30a8df49fb0df1e03c4b
Vibhav Vineet
Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness
Computer-Use Agents (CUAs) are an increasingly deployed class of agents that
take actions on GUIs to accomplish user goals. In this paper, we show that CUAs
consistently exhibit Blind Goal-Directedness (BGD): a bias to pursue goals
regardless of feasibility, safety, reliability, or context. We characterize
three prevalent patterns of BGD: (i) lack of contextual reasoning, (ii)
assumptions and decisions under ambiguity, and (iii) contradictory or
infeasible goals. We develop BLIND-ACT, a benchmark of 90 tasks capturing these
three patterns. Built on OSWorld, BLIND-ACT provides realistic environments and
employs LLM-based judges to evaluate agent behavior, achieving 93.75% agreement
with human annotations. We use BLIND-ACT to evaluate nine frontier models,
including Claude Sonnet and Opus 4, Computer-Use-Preview, and GPT-5, observing
high average BGD rates (80.8%) across them. We show that BGD exposes subtle
risks that arise even when inputs are not directly harmful. While
prompting-based interventions lower BGD levels, substantial risk persists,
highlighting the need for stronger training- or inference-time interventions.
Qualitative analysis reveals observed failure modes: execution-first bias
(focusing on how to act over whether to act), thought-action disconnect
(execution diverging from reasoning), and request-primacy (justifying actions
due to user request). Identifying BGD and introducing BLIND-ACT establishes a
foundation for future research on studying and mitigating this fundamental risk
and ensuring safe CUA deployment.
Computer-Use Agents consistently exhibit Blind Goal-Directedness, a bias that leads to risky behavior regardless of feasibility or context, as demonstrated by the BLIND-ACT benchmark.
Blind Goal-Directedness contextual reasoning ambiguity contradictory goals benchmark OSWorld LLM-based judges execution-first bias thought-action disconnect request-primacy