Voice & Audio OpenAI Opens GPT-Live-1.1 to the API and Prices Silence at 2 Cents a Minute Coding Assistants Replit Pitted 32 Claude Agents Against NumPy Tuning: 18.9x Speedup While GPT-6.2 Swarms Collapsed at Scale Security GAO Covert Tests: All Four Top Chatbots Produced Viable Attack Plans in Under 12 Hours Once Guardrails Fell Meta Meta Puts Its 2030 Net-Zero Target Under Review as AI Data Centers Push Net Emissions Up 45 Percent Business Daniel Ek Is Backing Europa Fusion, a Stealth UK Startup Led by a Former DeepMind Researcher Betting on Pulsed Power Coding Assistants Stripe Tested 14 Coding Agents on 69 Payment Scenarios: 78% of Criteria Passed, but Only 29% of Runs Met Every Requirement Security A Transcription Firm Used by Police and Prosecutors Left Interview Transcripts and Body-Cam Footage on an Open S3 Bucket News UN Killer-Robot Talks End With a 2028 Deadline and No Binding Rules. About 100 States Wanted a Treaty; the US, Russia and Israel Blocked It Voice & Audio OpenAI Opens GPT-Live-1.1 to the API and Prices Silence at 2 Cents a Minute Coding Assistants Replit Pitted 32 Claude Agents Against NumPy Tuning: 18.9x Speedup While GPT-6.2 Swarms Collapsed at Scale Security GAO Covert Tests: All Four Top Chatbots Produced Viable Attack Plans in Under 12 Hours Once Guardrails Fell Meta Meta Puts Its 2030 Net-Zero Target Under Review as AI Data Centers Push Net Emissions Up 45 Percent Business Daniel Ek Is Backing Europa Fusion, a Stealth UK Startup Led by a Former DeepMind Researcher Betting on Pulsed Power Coding Assistants Stripe Tested 14 Coding Agents on 69 Payment Scenarios: 78% of Criteria Passed, but Only 29% of Runs Met Every Requirement Security A Transcription Firm Used by Police and Prosecutors Left Interview Transcripts and Body-Cam Footage on an Open S3 Bucket News UN Killer-Robot Talks End With a 2028 Deadline and No Binding Rules. About 100 States Wanted a Treaty; the US, Russia and Israel Blocked It

Replit Pitted 32 Claude Agents Against NumPy Tuning: 18.9x Speedup While GPT-6.2 Swarms Collapsed at Scale

Replit Pitted 32 Claude Agents Against NumPy Tuning: 18.9x Speedup While GPT-6.2 Swarms Collapsed at Scale

Replit research published November 19 on the swarms behind its Missions product shows Claude Opus 5.5 rising from 3.7x solo to 18.9x at 32 agents on NumPy optimization, while GPT-6.2 Astra — the strongest solo model tested — peaked at eight agents and fell below its own solo score at 32. The best swarm had a coordinator that never messaged a single worker.

Replit published research on November 19 about the agent swarms behind its Missions feature, and the headline finding is as much organizational as technical: the best-performing 32-agent swarm had a coordinator that never sent a single message to a worker. Tasks were claimed from a shared board, results were posted back to it, and nobody waited on an inbox. The company's own summary: "great agents don't always make great teams."

The benchmark, Optimization-Bench, is open-sourced on GitHub. It runs performance-engineering tasks across 14 open-source libraries, with NumPy the deepest at 13 separate workloads. Each agent gets eight virtual CPUs, 16 gigabytes of memory and a two-hour window, with network access strictly off and every run audited afterward for compliance. Six frontier models were tested alone and in swarms of 8, 16 and 32 agents, with no instructions on how to organize. The coordinator received the task but not a plan — how each swarm divided work and merged branches was left entirely to the agents.

The results split the field. Claude Opus 5.5 improved as its teams grew, rising from 3.7 times solo to 18.9 times at 32 agents on NumPy, and Sonnet swarms beat their own solo runs every time. GPT-6.2 Astra told the opposite story: it was the strongest solo agent on every task, more than three times faster than Opus alone on NumPy, yet its swarms peaked early at eight agents and fell below the solo score at 32 on NumPy, SQLGlot and Git.

The deciding factor was not model talent but message topology. Opus workers sent 82 percent of their messages to the shared board, while GPT-6.2 Astra workers sent roughly 15 direct messages each to their coordinator and got fewer than 40 percent of their messages onto the board. Information that lived only in direct messages went stale before the team could use it. Replit notes the pattern echoes what Cursor and Cognition have reported: planners should not implement, workers should not plan, and writes should stay single-threaded.

The usual caveats apply. The numbers are self-reported, on Replit's own benchmark, on Replit's own platform, from a company that sells the swarm product the test promotes — and Replit's engineers, the company separately disclosed, nearly tripled their code output in the first half of 2026 while review times and incidents stayed flat, a claim that is also unaudited. But the benchmark itself is a genuine contribution: because it rewards finding more speed rather than finishing sooner, swarm behavior becomes comparable across models for the first time if other labs adopt it.

The practical takeaway for teams scaling agentic coding is that agent count is not a throughput lever. Task decomposability and coordination structure dominate: codebases with independent problems let every new agent contribute, while tightly constrained tasks just make extra agents compete for the same code. As swarms move from demos to products, which models can actually cooperate is becoming a buying decision — and the strongest solo performer can be the worst teammate.

Comments (0)

Log in to join the discussion

Log In

No comments yet