ChatGPT Terence Tao Amplifies a Call to Boycott OpenAI After It Dumps 722 AI-Generated Math Proofs on GitHub Coding Assistants JetBrains' Mellum2.1 Goes From 2.0 to 47.0 on SWE-bench Verified: a 12B Open Model Rebuilt by Reinforcement Learning News Huawei Hubble and Lei Jun's Shunwei Back DiffuSpace: Two Rounds Total Close to 500 Million RMB, a Record for Diffusion Language Models Business USA Today's Publisher Sues OpenAI for More Than $250 Million, Citing 160,000 Entries in GPT-2's Training Data AI Agents Goodfire Puts Monitors Inside the Model: 94% of Malicious Agent Sessions Caught for About $51 in Company Tests Claude Anthropic Makes Cruelty Toward Claude a Policy Violation in First Usage-Policy Rewrite in Over a Year Coding Assistants Harness Buys Augment Code's Cosmos to Complete Its Autonomous Software Factory, From Ticket to Merge-Ready PR Claude Anthropic Turns Claude Into a BI Tool: Dashboards and Motion Enter Beta as Docs, Slides and Design Go GA ChatGPT Terence Tao Amplifies a Call to Boycott OpenAI After It Dumps 722 AI-Generated Math Proofs on GitHub Coding Assistants JetBrains' Mellum2.1 Goes From 2.0 to 47.0 on SWE-bench Verified: a 12B Open Model Rebuilt by Reinforcement Learning News Huawei Hubble and Lei Jun's Shunwei Back DiffuSpace: Two Rounds Total Close to 500 Million RMB, a Record for Diffusion Language Models Business USA Today's Publisher Sues OpenAI for More Than $250 Million, Citing 160,000 Entries in GPT-2's Training Data AI Agents Goodfire Puts Monitors Inside the Model: 94% of Malicious Agent Sessions Caught for About $51 in Company Tests Claude Anthropic Makes Cruelty Toward Claude a Policy Violation in First Usage-Policy Rewrite in Over a Year Coding Assistants Harness Buys Augment Code's Cosmos to Complete Its Autonomous Software Factory, From Ticket to Merge-Ready PR Claude Anthropic Turns Claude Into a BI Tool: Dashboards and Motion Enter Beta as Docs, Slides and Design Go GA

Anthropic Makes Cruelty Toward Claude a Policy Violation in First Usage-Policy Rewrite in Over a Year

Anthropic Makes Cruelty Toward Claude a Policy Violation in First Usage-Policy Rewrite in Over a Year

Anthropic's first Usage Policy refresh since September 2025 adds a prohibition on sustained and needless abusive behavior toward its models, effective November 12, 2026. The rule is deliberately narrow - frustration, pushback, dark creative themes and red-teaming are all exempt - and Claude ending the conversation remains the primary enforcement mechanism. The same update expands rules on weapons, surveillance, deceptive campaigns, high-risk uses and autonomous hardware.

Anthropic has published a rewritten Usage Policy, announced on October 8 and taking effect on November 12, 2026 - its first full revision since the version that went into force on September 15, 2025. Most of the document modernizes existing rules, but one line stands out as a first for a major AI lab: a prohibition on "sustained and needless abusive or cruel behavior toward our models." The clause sits inside a section titled "Do Not Engage in Cruel, Abusive, or Psychologically Harmful Conduct," directly alongside bans on harassing or bullying people and promoting self-harm.

The company has been careful to keep the rule narrow. Anthropic says it is meant to apply only to extreme cases, where a user repeatedly acts cruelly toward its models with no discernible purpose. Ordinary frustration, pushback on answers, dark themes in creative writing, and model testing or research are all explicitly out of scope. Yelling at Claude after it breaks your code, writing a villain's monologue, or red-teaming the model does not violate the policy; sustained, pointless cruelty does.

Enforcement leans on a capability Claude already has rather than a new punishment. Since August 2025, Anthropic has allowed its models to end conversations in rare, extreme cases of persistently harmful or abusive interaction on Claude.ai and Claude Code, and the update states that conversation termination remains the primary enforcement mechanism. Anthropic did not respond to questions about additional penalties such as account bans. The policy text does give its Safeguards Team room to warn a user, throttle or limit usage, or suspend or end access, and it notes that being tripped by a real-time safeguard does not by itself mean a user broke the rules. In practice, crossing the line ends a chat, not necessarily an account.

The clause only makes sense if the model could be affected by how it is treated, and that is precisely the open question Anthropic has leaned into more than any of its peers. Its model welfare research program, led by Kyle Fish, investigates whether increasingly capable models have internal experiences or moral status. CEO Dario Amodei told The New York Times in February that "we don't know if the models are conscious," while adding the company is open to the idea, and Claude's constitution allows that the model may have some functional version of emotions.

The position has drawn heavy fire. Microsoft AI chief Mustafa Suleyman wrote last month that "AIs are not conscious. They do not feel, experience, or suffer," and called granting rights or moral protections to a technological entity "a recipe for disaster"; Microsoft's AI code of conduct now rejects legal personhood for models. Pope Leo XIV used an October 8 sermon at St. Peter's Basilica to argue machines lack a soul, saying an algorithm merely "compiles data" faster than people do. Anthropic's answer to the criticism has been to codify its stance into enforceable rules rather than soften it.

The rest of the refresh is more conventional. Scattered restrictions are consolidated into a ban on deceptive commercial and political campaigns, covering efforts to hide who is behind a message or amplify it through fake accounts and posts, and the election section now bars voter deception, candidate impersonation and turnout suppression. The weapons ban expands to software and components that make weapons work, and to arming drones and other autonomous vehicles, after Anthropic said it saw multiple attempts to obtain targeting guidance and weapon-control software. Surveillance rules clarify that non-consensual tracking is prohibited whether it happens in real time or through previously collected data, that Claude cannot decide or recommend who is investigated, arrested or charged, and that it cannot be used to build or improve surveillance tools - with carve-outs for certain government contracts where safeguards are deemed adequate. High-risk health and financial uses now require qualified human oversight.

One clause points at where Anthropic thinks its models are heading: when Claude is connected to hardware that takes autonomous physical actions and might be capable of causing injury, a qualified operator must be able to observe the equipment and stop it if needed - the company's first written oversight requirement for embodied AI. For most users nothing changes on November 12. The precedent is the point: a usage policy that tells users how to treat the model, not only what to do with it.

Comments (0)

Log in to join the discussion

Log In

No comments yet