Replit published research on November 19 about the agent swarms behind its Missions feature, and the headline finding is as much organizational as technical: the best-performing 32-agent swarm had a coordinator that never sent a single message to a worker. Tasks were claimed from a shared board, results were posted back to it, and nobody waited on an inbox. The company's own summary: "great agents don't always make great teams."
The benchmark, Optimization-Bench, is open-sourced on GitHub. It runs performance-engineering tasks across 14 open-source libraries, with NumPy the deepest at 13 separate workloads. Each agent gets eight virtual CPUs, 16 gigabytes of memory and a two-hour window, with network access strictly off and every run audited afterward for compliance. Six frontier models were tested alone and in swarms of 8, 16 and 32 agents, with no instructions on how to organize. The coordinator received the task but not a plan — how each swarm divided work and merged branches was left entirely to the agents.
The results split the field. Claude Opus 5.5 improved as its teams grew, rising from 3.7 times solo to 18.9 times at 32 agents on NumPy, and Sonnet swarms beat their own solo runs every time. GPT-6.2 Astra told the opposite story: it was the strongest solo agent on every task, more than three times faster than Opus alone on NumPy, yet its swarms peaked early at eight agents and fell below the solo score at 32 on NumPy, SQLGlot and Git.
The deciding factor was not model talent but message topology. Opus workers sent 82 percent of their messages to the shared board, while GPT-6.2 Astra workers sent roughly 15 direct messages each to their coordinator and got fewer than 40 percent of their messages onto the board. Information that lived only in direct messages went stale before the team could use it. Replit notes the pattern echoes what Cursor and Cognition have reported: planners should not implement, workers should not plan, and writes should stay single-threaded.
The usual caveats apply. The numbers are self-reported, on Replit's own benchmark, on Replit's own platform, from a company that sells the swarm product the test promotes — and Replit's engineers, the company separately disclosed, nearly tripled their code output in the first half of 2026 while review times and incidents stayed flat, a claim that is also unaudited. But the benchmark itself is a genuine contribution: because it rewards finding more speed rather than finishing sooner, swarm behavior becomes comparable across models for the first time if other labs adopt it.
The practical takeaway for teams scaling agentic coding is that agent count is not a throughput lever. Task decomposability and coordination structure dominate: codebases with independent problems let every new agent contribute, while tightly constrained tasks just make extra agents compete for the same code. As swarms move from demos to products, which models can actually cooperate is becoming a buying decision — and the strongest solo performer can be the worst teammate.
Comments (0)
Log in to join the discussion
Log InNo comments yet