FlashAdventure: A Benchmark for GUI Agents Solving Full Story Arcs in
  Diverse Adventure Games
GUI agents powered by LLMs show promise in interacting with diverse digital
environments. Among these, video games offer a valuable testbed due to their
varied interfaces, with adventure games posing additional challenges through
complex, narrative-driven interactions. Existing game benchmarks, however, lack
diversity and rarely evaluate agents on completing entire storylines. To
address this, we introduce FlashAdventure, a benchmark of 34 Flash-based
adventure games designed to test full story arc completion and tackle the
observation-behavior gap: the challenge of remembering and acting on earlier
gameplay information. We also propose CUA-as-a-Judge, an automated gameplay
evaluator, and COAST, an agentic framework leveraging long-term clue memory to
better plan and solve sequential tasks. Experiments show current GUI agents
struggle with full story arcs, while COAST improves milestone completion by
bridging the observation-behavior gap. Nonetheless, a marked discrepancy
between humans and best-performing agents warrants continued research efforts
to narrow this divide.
68b7c8db295f15ff60911459
Jaewoo Ahn
claimed_verified
2025-09-03T08:26:17.784Z
64bb081c01f1983a863654dc
/avatars/038894bc72b92ec3f4ecb096cc60b60a.svg
Jaewoo Ahn
user
ahnpersie
68b7c8db295f15ff6091145a
Junseo Kim
68b7c8db295f15ff6091145b
Heeseung Yun
68b7c8db295f15ff6091145c
Jaehyeon Son
68b7c8db295f15ff6091145d
Dongmin Park
68b7c8db295f15ff6091145e
Jaewoong Cho
68b7c8db295f15ff6091145f
Gunhee Kim
FlashAdventure: A Benchmark for GUI Agents Solving Full Story Arcs in
  Diverse Adventure Games
GUI agents powered by LLMs show promise in interacting with diverse digital
environments. Among these, video games offer a valuable testbed due to their
varied interfaces, with adventure games posing additional challenges through
complex, narrative-driven interactions. Existing game benchmarks, however, lack
diversity and rarely evaluate agents on completing entire storylines. To
address this, we introduce FlashAdventure, a benchmark of 34 Flash-based
adventure games designed to test full story arc completion and tackle the
observation-behavior gap: the challenge of remembering and acting on earlier
gameplay information. We also propose CUA-as-a-Judge, an automated gameplay
evaluator, and COAST, an agentic framework leveraging long-term clue memory to
better plan and solve sequential tasks. Experiments show current GUI agents
struggle with full story arcs, while COAST improves milestone completion by
bridging the observation-behavior gap. Nonetheless, a marked discrepancy
between humans and best-performing agents warrants continued research efforts
to narrow this divide.
FlashAdventure benchmark and COAST framework improve GUI agents' performance in completing full story arcs in Flash-based adventure games by addressing the observation-behavior gap.
LLMs GUI agents FlashAdventure CUA-as-a-Judge COAST long-term clue memory observation-behavior gap