StockBench: Can LLM Agents Trade Stocks Profitably In Real-world
  Markets?
Large language models (LLMs) have recently demonstrated strong capabilities
as autonomous agents, showing promise in reasoning, tool use, and sequential
decision-making. While prior benchmarks have evaluated LLM agents in domains
such as software engineering and scientific discovery, the finance domain
remains underexplored, despite its direct relevance to economic value and
high-stakes decision-making. Existing financial benchmarks primarily test
static knowledge through question answering, but they fall short of capturing
the dynamic and iterative nature of trading. To address this gap, we introduce
StockBench, a contamination-free benchmark designed to evaluate LLM agents in
realistic, multi-month stock trading environments. Agents receive daily market
signals -- including prices, fundamentals, and news -- and must make sequential
buy, sell, or hold decisions. Performance is assessed using financial metrics
such as cumulative return, maximum drawdown, and the Sortino ratio. Our
evaluation of state-of-the-art proprietary (e.g., GPT-5, Claude-4) and
open-weight (e.g., Qwen3, Kimi-K2, GLM-4.5) models shows that while most LLM
agents struggle to outperform the simple buy-and-hold baseline, several models
demonstrate the potential to deliver higher returns and manage risk more
effectively. These findings highlight both the challenges and opportunities in
developing LLM-powered financial agents, showing that excelling at static
financial knowledge tasks does not necessarily translate into successful
trading strategies. We release StockBench as an open-source resource to support
reproducibility and advance future research in this domain.
68df3031df49fb0df1e03c3a
Yanxu Chen
claimed_verified
2025-10-10T09:39:14.977Z
66495ca132236a0fad5d3124
https://cdn-avatars.huggingface.co/v1/production/uploads/66495ca132236a0fad5d3124/M_hHgHtV0n2OCV28PKRr9.jpeg
Chen Yanxu
user
CCCCCyx
68df3031df49fb0df1e03c3b
Zijun Yao
68df3031df49fb0df1e03c3c
Yantao Liu
68df3031df49fb0df1e03c3d
Jin Ye
68df3031df49fb0df1e03c3e
Jianing Yu
68df3031df49fb0df1e03c3f
Lei Hou
68df3031df49fb0df1e03c40
Juanzi Li
StockBench: Can LLM Agents Trade Stocks Profitably In Real-world
  Markets?
Large language models (LLMs) have recently demonstrated strong capabilities
as autonomous agents, showing promise in reasoning, tool use, and sequential
decision-making. While prior benchmarks have evaluated LLM agents in domains
such as software engineering and scientific discovery, the finance domain
remains underexplored, despite its direct relevance to economic value and
high-stakes decision-making. Existing financial benchmarks primarily test
static knowledge through question answering, but they fall short of capturing
the dynamic and iterative nature of trading. To address this gap, we introduce
StockBench, a contamination-free benchmark designed to evaluate LLM agents in
realistic, multi-month stock trading environments. Agents receive daily market
signals -- including prices, fundamentals, and news -- and must make sequential
buy, sell, or hold decisions. Performance is assessed using financial metrics
such as cumulative return, maximum drawdown, and the Sortino ratio. Our
evaluation of state-of-the-art proprietary (e.g., GPT-5, Claude-4) and
open-weight (e.g., Qwen3, Kimi-K2, GLM-4.5) models shows that while most LLM
agents struggle to outperform the simple buy-and-hold baseline, several models
demonstrate the potential to deliver higher returns and manage risk more
effectively. These findings highlight both the challenges and opportunities in
developing LLM-powered financial agents, showing that excelling at static
financial knowledge tasks does not necessarily translate into successful
trading strategies. We release StockBench as an open-source resource to support
reproducibility and advance future research in this domain.
StockBench evaluates large language models in realistic stock trading environments, revealing challenges and opportunities in developing LLM-powered financial agents.
large language models LLM agents reasoning tool use sequential decision-making software engineering scientific discovery finance domain financial benchmarks question answering dynamic trading contamination-free benchmark daily market signals prices fundamentals news sequential buy sell hold decisions cumulative return maximum drawdown Sortino ratio GPT-5 Claude-4 Qwen3 Kimi-K2 GLM-4.5 buy-and-hold baseline static financial knowledge tasks trading strategies