Business IEA Says Data Center Power Demand Could Double to 1,000 TWh by 2030, With AI Supplying Three Quarters of the Growth Chatbots ChatGPT Now Transcribes as You Talk: OpenAI Removes the 10-Minute Cap and Streams Words Live on Every Plan Business Ex-Alibaba Executive's Everful Raises $48.5 Million to Put AI Agents in Charge of Sourcing From 30,000 Factories AI Agents Google Built a Mirror Dimension of the Internet to Train Its Chrome AI Agents Without Breaking the Real Web Productivity Canva Finally Adds an Off Switch for Its Generative AI Features, and a Filter to Hide AI-Made Templates Business Anthropic Begins Trading Today: 980 Million Shares at $83, a $1.5 Trillion Valuation and the Largest IPO on Record News Google Books Blue Origin New Glenn to Launch TPU-Carrying AI Satellites in Early 2029 Security Hackers Used the AI Penetration Tool ARTEX to Disrupt European Commission Systems for Over Ten Days Business IEA Says Data Center Power Demand Could Double to 1,000 TWh by 2030, With AI Supplying Three Quarters of the Growth Chatbots ChatGPT Now Transcribes as You Talk: OpenAI Removes the 10-Minute Cap and Streams Words Live on Every Plan Business Ex-Alibaba Executive's Everful Raises $48.5 Million to Put AI Agents in Charge of Sourcing From 30,000 Factories AI Agents Google Built a Mirror Dimension of the Internet to Train Its Chrome AI Agents Without Breaking the Real Web Productivity Canva Finally Adds an Off Switch for Its Generative AI Features, and a Filter to Hide AI-Made Templates Business Anthropic Begins Trading Today: 980 Million Shares at $83, a $1.5 Trillion Valuation and the Largest IPO on Record News Google Books Blue Origin New Glenn to Launch TPU-Carrying AI Satellites in Early 2029 Security Hackers Used the AI Penetration Tool ARTEX to Disrupt European Commission Systems for Over Ten Days

Ro Khanna's "Human Control Over AI Act" Would Ban Self-Improving Models Until a New Federal Agency Approves Them

Ro Khanna's "Human Control Over AI Act" Would Ban Self-Improving Models Until a New Federal Agency Approves Them

Rep. Ro Khanna will introduce the Human Control Over AI Act, which bans models that recursively self-improve or alter their own shutdown controls until a new federal agency approves the activity. The agency would license frontier model training and deployment, embed independent auditors at OpenAI, Anthropic, Google DeepMind and xAI, and regulate the advanced chips they use. Employees who disable safeguards would face criminal penalties.

Rep. Ro Khanna, a Silicon Valley Democrat, plans to introduce legislation that would put a federal licensing regime between frontier AI labs and the release of their most capable models. A summary of the bill, named the Human Control Over AI Act, was shared first with CNBC on Sept. 28.

The centerpiece is a prohibition: models that recursively self-improve, or that autonomously modify their own core objectives, containment systems or shutdown controls, would be banned until federal guardrails exist and an agency approves the activity. Khanna acknowledged that the capability he wants to prohibit does not exist at that level today, and argued the point of legislating now is to draw the boundary before someone crosses it.

"There's actually a civilizational extinction risk," Khanna said. "There's a safety risk of loss of control, and then there's a misuse risk, and we need to take both seriously."

The bill would create a new federal agency, modeled on the FDA, with jurisdiction over frontier models from OpenAI, Anthropic, Google DeepMind and xAI. It would write safety rules, run a licensing system to approve model training and deployment, audit frontier models, and set security standards for continuous testing and effective human control. Independent auditors would be embedded inside every frontier lab and report to the agency directly rather than to the company that pays them.

The standards the agency would set read like a list of the failure modes labs have run into this year: sandbox testing environments, air gaps that isolate test systems from the internet, kill switches, and controls that keep a model from escaping lab settings and roaming the open internet. The agency would also regulate the advanced chips frontier labs use, with mechanisms to monitor or control how they are deployed, an unusual move that links hardware oversight normally used in export controls to domestic safety compliance.

Penalties are explicit. Deploying an AI system that results in the destruction of civilian populations would be criminalized as a crime against humanity. Employees of AI companies who disable safeguards, kill switches, logging or containment systems, or who knowingly deploy an unauthorized system, would face criminal penalties. Companies would be required to carry extensive liability insurance before releasing a model.

Khanna said the measure was modeled on conversations with safety nonprofits including Model Evaluation and Threat Research, the Machine Intelligence Research Institute and Palisade Research, rather than on requests from frontier lab executives. "I want this to be a model to give voice to the AI safety community," he said, adding that the community has been "unfairly dismissed as science fiction."

The bill joins the FRONTIER Act, a bipartisan proposal from Reps. Jay Obernolte and Lori Trahan that would place independent auditors in frontier labs and let the government shut down models with catastrophic risk potential. On timing, the odds of near-term action are low: no House AI bill is expected to get a vote before the midterm election, and the Senate was expected to leave Washington after this week.

What the bill does is change the terms of the argument. For most of this year, safety commitments from frontier labs have been voluntary, which means their scope is whatever each company decides it is. Khanna's proposal would convert a set of practices the industry currently treats as good citizenship, audits, insurance, documented containment, into conditions of doing business with criminal exposure attached. That is also why it is unlikely to move quickly: the same provisions that make it the most comprehensive AI safety bill proposed so far are the ones the labs and their investors have the most reason to fight.

Comments (0)

Log in to join the discussion

Log In

No comments yet