AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs
Large-language models (LLMs) have demonstrated powerful problem-solving
capabilities, in particular when organized in multi-agent systems. However, the
advent of such systems also raises several questions on the ability of a
complex network of agents to effectively self-organize and collaborate. While
measuring performance on standard reasoning benchmarks indicates how well
multi-agent systems can solve reasoning tasks, it is unclear whether these
systems are able to leverage their topology effectively. Here, we propose
AgentsNet, a new benchmark for multi-agent reasoning. By drawing inspiration
from classical problems in distributed systems and graph theory, AgentsNet
measures the ability of multi-agent systems to collaboratively form strategies
for problem-solving, self-organization, and effective communication given a
network topology. We evaluate a variety of baseline methods on AgentsNet
including homogeneous networks of agents which first have to agree on basic
protocols for organization and communication. We find that some frontier LLMs
are already demonstrating strong performance for small networks but begin to
fall off once the size of the network scales. While existing multi-agent
benchmarks cover at most 2-5 agents, AgentsNet is practically unlimited in size
and can scale with new generations of LLMs. As such, we also probe frontier
models in a setup with up to 100 agents.
687764e0ff8f47a7f86442ad
Florian Grötschla
687764e0ff8f47a7f86442ae
Luis Müller
687764e0ff8f47a7f86442af
Jan Tönshoff
687764e0ff8f47a7f86442b0
Mikhail Galkin
687764e0ff8f47a7f86442b1
Bryan Perozzi
AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs
Large-language models (LLMs) have demonstrated powerful problem-solving
capabilities, in particular when organized in multi-agent systems. However, the
advent of such systems also raises several questions on the ability of a
complex network of agents to effectively self-organize and collaborate. While
measuring performance on standard reasoning benchmarks indicates how well
multi-agent systems can solve reasoning tasks, it is unclear whether these
systems are able to leverage their topology effectively. Here, we propose
AgentsNet, a new benchmark for multi-agent reasoning. By drawing inspiration
from classical problems in distributed systems and graph theory, AgentsNet
measures the ability of multi-agent systems to collaboratively form strategies
for problem-solving, self-organization, and effective communication given a
network topology. We evaluate a variety of baseline methods on AgentsNet
including homogeneous networks of agents which first have to agree on basic
protocols for organization and communication. We find that some frontier LLMs
are already demonstrating strong performance for small networks but begin to
fall off once the size of the network scales. While existing multi-agent
benchmarks cover at most 2-5 agents, AgentsNet is practically unlimited in size
and can scale with new generations of LLMs. As such, we also probe frontier
models in a setup with up to 100 agents.
AgentsNet is a new benchmark for evaluating multi-agent systems' ability to self-organize, communicate, and solve problems collaboratively across varying network sizes.
multi-agent systems AgentsNet distributed systems graph theory self-organization communication network topology homogeneous networks protocols LLMs