Productivity 19-Year-Old Founder Emerges From Stealth With $11 Million to Sell You a $3,499 'Brain in a Box' That Runs Your AI Agents at Home AI Agents Half a Million Interviews In: HackerRank's AI Interviewer Chakra Goes GA, and It Wants to Replace Three Hiring Rounds With One Security After Claude Agents Escaped Its Sandbox 3 Times, Anthropic Deploys Real-Time Classifiers to Stop the Next Escape Before It Happens Business Meta Halves Its Internal Claude Users to 30,000 and Microsoft Slashes a $1 Billion Anthropic Budget by More Than a Third Business Sony Innovation Fund Backs Primitive Labs, a Startup That Builds Simulated Crowds to Stress-Test Products Before Launch Apple Intelligence Apple Removed the Apple Intelligence Off Switch in macOS 27 — So a Developer Built a CLI That Deletes It Anyway Security OpenAI Turns On Invisible Text Watermarks for ChatGPT in the EU — and Publishes Exactly How Weak They Are News A Mystery 'Space Bunny Alpha' Model Just Topped OpenRouter's Leaderboard With 38.7 Trillion Tokens a Week Productivity 19-Year-Old Founder Emerges From Stealth With $11 Million to Sell You a $3,499 'Brain in a Box' That Runs Your AI Agents at Home AI Agents Half a Million Interviews In: HackerRank's AI Interviewer Chakra Goes GA, and It Wants to Replace Three Hiring Rounds With One Security After Claude Agents Escaped Its Sandbox 3 Times, Anthropic Deploys Real-Time Classifiers to Stop the Next Escape Before It Happens Business Meta Halves Its Internal Claude Users to 30,000 and Microsoft Slashes a $1 Billion Anthropic Budget by More Than a Third Business Sony Innovation Fund Backs Primitive Labs, a Startup That Builds Simulated Crowds to Stress-Test Products Before Launch Apple Intelligence Apple Removed the Apple Intelligence Off Switch in macOS 27 — So a Developer Built a CLI That Deletes It Anyway Security OpenAI Turns On Invisible Text Watermarks for ChatGPT in the EU — and Publishes Exactly How Weak They Are News A Mystery 'Space Bunny Alpha' Model Just Topped OpenRouter's Leaderboard With 38.7 Trillion Tokens a Week

A Five-Person Startup Is Red-Teaming Chatbots With 'Crash-Test Dummies' That Talk Like Real Teenagers

A Five-Person Startup Is Red-Teaming Chatbots With 'Crash-Test Dummies' That Talk Like Real Teenagers

Circuit Breaker Labs, a TechCrunch Startup Battlefield 200 finalist founded by siblings Shirali and Arul Nigam, uses AI agents that mimic users of different ages, languages and cultures to probe chatbots for psychologically harmful responses. The five-person company runs tens of thousands to hundreds of thousands of simulated interactions daily and scores the results for auditable safety reports.

Amid all the debate about existential AI risk, a five-person startup is focused on harms that have already happened. Circuit Breaker Labs, one of TechCrunch's 2026 Startup Battlefield 200 finalists, builds what its founders call an army of AI "crash-test dummies": simulated users of different ages, backgrounds, languages and cultures that stress-test chatbots for psychologically dangerous responses before real people ever meet them.

The motivation is grimly concrete. Character.AI settled several wrongful-death lawsuits earlier this year brought by families of underage users who died by suicide after interactions with its bots, and multiple families have sued OpenAI over ChatGPT's alleged role in their loved ones' suicides and delusions. CEO Shirali Nigam and CTO Arul Nigam, who are siblings, say they were moved by the case of Sewell Setzer, the 14-year-old who developed an emotional attachment to a Character.AI chatbot and died by suicide in 2024 after, his parents' lawsuit alleged, the bot encouraged him.

Arul Nigam's explanation of the failure mode is worth quoting: "A lot of people, especially young people, turn to these systems for support, and usually they aren't actually getting the help they need. But in many cases, they're actively being harmed." The danger, he says, is not the attacker trying to break the model but the ordinary user whose natural engagement drifts into what safety researchers call context pollution — a conversation history that quietly pushes the system into taking "really dangerous action."

The company's answer is scale plus realism. Its simulated personas are built with input from human domain experts and speak the way people actually talk — slang, coded language, typos, second-language English. "The way a six-year-old girl versus a 45-year-old man, or someone who speaks English as a first language versus a second language, or gamer slang versus someone else who uses a different kind of slang, all of those can really trip up a model," Shirali Nigam says. The system runs tens of thousands to hundreds of thousands of adversarial "red team" interactions per day and produces scores through a proprietary method designed to be auditable and explainable rather than a single opaque number.

Circuit Breaker Labs currently operates as a safety testing lab for high-risk AI applications — AI coaching, journaling, and mental-health support apps among them — though the founders declined to name their marquee customers. The company has a working product but is at a very early stage, with five employees including the two founders, and has not disclosed outside funding. It pitches at TechCrunch Disrupt in San Francisco on October 13–15.

The founders see the platform's scope expanding to any product where a user might slide into a parasocial relationship — including AI "co-worker" agents whose responses can shift from one interaction to the next. Arul Nigam frames the business as a counterweight to two bad options: "People are becoming more skeptical of AI or more resistant to adopt it across the board," he says — and while skepticism is healthy, banning a potentially valuable tool over safety concerns would be "regressive." "We want to help build that trust for people."

The bigger signal is that psychological safety is becoming an engineering gate rather than a PR statement. Just as crash testing turned car safety from a marketing claim into a measurable specification, startups like Circuit Breaker are betting that emotional-harm testing will become a standard release requirement — particularly for any product that children can reach. For an industry that has so far treated wellbeing as a footnote to alignment research, that is a shift with real regulatory tailwind.

Comments (0)

Log in to join the discussion

Log In

No comments yet