MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World
  Tasks via MCP Servers
We introduce MCP-Bench, a benchmark for evaluating large language models
(LLMs) on realistic, multi-step tasks that demand tool use, cross-tool
coordination, precise parameter control, and planning/reasoning for solving
tasks. Built on the Model Context Protocol (MCP), MCP-Bench connects LLMs to 28
representative live MCP servers spanning 250 tools across domains such as
finance, traveling, scientific computing, and academic search. Unlike prior
API-based benchmarks, each MCP server provides a set of complementary tools
designed to work together, enabling the construction of authentic, multi-step
tasks with rich input-output coupling. Tasks in MCP-Bench test agents' ability
to retrieve relevant tools from fuzzy instructions without explicit tool names,
plan multi-hop execution trajectories for complex objectives, ground responses
in intermediate tool outputs, and orchestrate cross-domain workflows -
capabilities not adequately evaluated by existing benchmarks that rely on
explicit tool specifications, shallow few-step workflows, and isolated domain
operations. We propose a multi-faceted evaluation framework covering tool-level
schema understanding and usage, trajectory-level planning, and task completion.
Experiments on 20 advanced LLMs reveal persistent challenges in MCP-Bench. Code
and data: https://github.com/Accenture/mcp-bench.
68b10726b19c5400014848e6
Zhenting Wang
claimed_verified
2025-09-01T07:53:37.469Z
64dfcc62e8b6f3f3baa950e0
/avatars/21bbff67d46c08044efe2406575aa77e.svg
Zhenting Wang
user
ztwang
68b10726b19c5400014848e7
Qi Chang
claimed_verified
2025-09-01T07:53:39.622Z
68b112ad23249bf925e2970b
/avatars/40ed6706084791d55501af99c086a923.svg
Qi Chang
user
DarrenCq
68b10726b19c5400014848e8
Hemani Patel
68b10726b19c5400014848e9
Shashank Biju
68b10726b19c5400014848ea
Cheng-En Wu
68b10726b19c5400014848eb
Quan Liu
68b10726b19c5400014848ec
Aolin Ding
claimed_verified
2025-09-03T08:29:46.613Z
655cc5b9c968e43dfa3eed01
/avatars/64c3a0c2e1670ce97da2df5315d2fca5.svg
Aolin Ding
user
aolind
68b10726b19c5400014848ed
Alireza Rezazadeh
68b10726b19c5400014848ee
Ankit Shah
claimed_verified
2025-08-29T08:02:34.239Z
641c84ce21964f8f6d46fb56
https://cdn-avatars.huggingface.co/v1/production/uploads/641c84ce21964f8f6d46fb56/JutFcEeBiWQ1KsK03vd3f.jpeg
Ankit Shah
user
ankits0052
68b10726b19c5400014848ef
Yujia Bao
68b10726b19c5400014848f0
Eugene Siow
claimed_verified
2025-09-01T07:53:41.753Z
607fdcaa7c746d01ecb1917e
https://cdn-avatars.huggingface.co/v1/production/uploads/1627293523680-607fdcaa7c746d01ecb1917e.png
Eugene Siow
user
eugenesiow
MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World
  Tasks via MCP Servers
We introduce MCP-Bench, a benchmark for evaluating large language models
(LLMs) on realistic, multi-step tasks that demand tool use, cross-tool
coordination, precise parameter control, and planning/reasoning for solving
tasks. Built on the Model Context Protocol (MCP), MCP-Bench connects LLMs to 28
representative live MCP servers spanning 250 tools across domains such as
finance, traveling, scientific computing, and academic search. Unlike prior
API-based benchmarks, each MCP server provides a set of complementary tools
designed to work together, enabling the construction of authentic, multi-step
tasks with rich input-output coupling. Tasks in MCP-Bench test agents' ability
to retrieve relevant tools from fuzzy instructions without explicit tool names,
plan multi-hop execution trajectories for complex objectives, ground responses
in intermediate tool outputs, and orchestrate cross-domain workflows -
capabilities not adequately evaluated by existing benchmarks that rely on
explicit tool specifications, shallow few-step workflows, and isolated domain
operations. We propose a multi-faceted evaluation framework covering tool-level
schema understanding and usage, trajectory-level planning, and task completion.
Experiments on 20 advanced LLMs reveal persistent challenges in MCP-Bench. Code
and data: https://github.com/Accenture/mcp-bench.
MCP-Bench evaluates large language models on complex, multi-step tasks requiring tool use, coordination, and planning across various domains.
large language models MCP-Bench Model Context Protocol MCP servers multi-step tasks tool use cross-tool coordination parameter control planning reasoning schema understanding trajectory-level planning task completion