Home · Technology · Oct 9 archive

Anthropic Bans 'Cruel' Treatment of AI Systems

12:05 PM ETConfirmed

Technology Desk

In Short: Despite the lack of explicit mention of 'model welfare,' both publications agree that Anthropic's policy update reflects growing concerns about the ethical treatment of AI systems.

The BBC (United Kingdom) and The Express Tribune (Pakistan) reported that Anthropic has introduced a new policy prohibiting users from engaging in sustained and needless abusive behavior towards its AI systems, including Claude.

According to the BBC (United Kingdom), Anthropic stated that the policy would only apply in extreme cases of repeated abuse, with no discernible purpose, and would not affect common user frustrations, model testing, or dark creative themes.

The Express Tribune (Pakistan) noted that Anthropic's policy update, which takes effect on November 12, includes provisions to prevent election interference, weapons development, and surveillance, alongside the ban on abusive behavior.

Both outlets agree that Anthropic's new policy allows its AI models to terminate conversations with users who are persistently harmful or abusive, with the BBC (United Kingdom) emphasizing that this remains the primary enforcement method.

The BBC (United Kingdom) reported that Anthropic's policy update has reignited debate over how we should communicate with AI tools, while The Express Tribune (Pakistan) highlighted that the policy does not explicitly cite 'model welfare,' though Anthropic has entertained the concept.

Despite the lack of explicit mention of 'model welfare,' both publications agree that Anthropic's policy update reflects growing concerns about the ethical treatment of AI systems.

The BBC (United Kingdom) and The Express Tribune (Pakistan) both noted that the policy update has been widely shared and discussed on social media, with some praising it as advancing good manners or 'model welfare.'

The policy update, according to The Express Tribune (Pakistan), does not grant legal personhood or rights to AI models, but it does suggest that Anthropic is considering the welfare of its AI systems.

The BBC (United Kingdom) reported that Anthropic's decision follows months of internal discussions about AI welfare, while The Express Tribune (Pakistan) highlighted that the policy update is designed for extreme situations rather than ordinary conversations.

Both publications agree that the new policy update aims to prevent sustained abusive behavior towards AI models, with the BBC (United Kingdom) emphasizing the importance of good manners and The Express Tribune (Pakistan) focusing on the ethical considerations of AI welfare.

The BBC (United Kingdom) and The Express Tribune (Pakistan) agree that Anthropic's policy update represents a significant step in regulating how users interact with AI systems, reflecting broader industry debates on AI ethics and governance.

What this adds

Background

What's confirmed

• The company said it would not apply to "common versions of user frustration, pushback, dark creative themes, or model testing and research" - suggesting it would only apply in clearly deliberate instances.
• Anthropic has announced an update to its usage policy, aiming to prevent "cruel behavior" towards its AI system, Claude.
• The AI company has equipped its language models with the ability to end interactions that are "potentially distressing." According to Storyboard18, Anthropic's Claude Opus 4 and 4.1 can now exit conversations where users are abusive or persistently harmful.
• Anthropic said Claude’s ability to end these conversations remains the main enforcement method.
• Anthropic, the company behind Claude, updated its usage policy and is giving users just over a month to get any lingering insults out of their systems.
• The changes take effect Nov. 12, and the update is “meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose,” Anthropic said.
• The updated policy does not explicitly cite so-called "model welfare," the idea that AI systems might deserve types of protection usually reserved for living things, but Anthropic and its executives have openly entertained the concept.
• Anthropic's decision follows months of internal discussions about AI welfare, according to sources familiar with the company's thinking.
• Anthropic Bans 'Cruel' Treatment of Claude AI Systems AI company introduces first-of-its-kind policy against abusive user behavior PUBLISHED: Fri, Oct 9, 2026, 1:37 PM UTC | UPDATED: Fri, Oct 9, 2026, 1:37 PM UTC Anthropic just made AI history by becoming the first major company to explicitly ban users from being "cruel" to its artificial intelligence systems.
• The updated policy, which takes effect November 12, also adds new restrictions on election interference, weapons software, and surveillance Anthropic published an updated usage policy on Thursday that prohibits sustained abusive or cruel behavior toward its Claude AI models, alongside new or clarified restrictions on election interference, weapons development, and surveillance.
• On elections, Anthropic consolidated several existing rules into a section titled "Do Not Undermine Democratic Processes," which bars using Claude to spread false information about candidates or voting, impersonate candidates or election officials, or attempt to suppress turnout.

What's still developing

• When a X user last year asked about electricity costs incurred by ChatGPT replying to users' "please" and "thank you" messages, OpenAI boss Sam Altman said it was, external "tens of millions of dollars well spent - you never know".
• Users will still be allowed to challenge Claude’s answers, test its limits, express anger when responses are poor, and explore dark themes in creative writing.
• Anthropic has also employed researchers focused on AI welfare, a field that examines how advanced systems should be treated and whether their development raises questions about possible suffering.
• At the time, Anthropic said it was unsure whether Claude has moral status, but the company wanted low-cost ways to reduce risks to Claude.
• Anthropic found that Claude Opus 4 had a "robust and consistent aversion to harm." The model demonstrated a preference against dealing with dangerous tasks, apparent distress when speaking with users seeking abusive conversations, and a tendency to end harmful conversations when allowed to do so.
• One of the most unusual changes focuses on how users treat Claude.
• Anthropic reckons attackers will have the advantage for the next two years before AI-powered defenses begin to catch up.
• Anthropic has tightened its restrictions on deceptive influence campaigns after observing state media outlets, government propaganda offices, and commercial organizations using Claude to operate fake accounts and fabricated news websites.
• Its election section also prohibits voter deception and election disruption, including spreading misinformation “about candidates or how to vote, impersonating candidates or election officials, or trying to suppress turnout,” Anthropic wrote.
• The Claude-maker said on Thursday it was updating its usage policy to let its tools end interactions where people are "cruel" - something it said it had done in "rare" cases prior.
• "We've further sharpened our section on elections, focusing it specifically on disallowing the use of Claude to deceive voters or disrupt elections (for instance, by spreading false information about candidates or how to vote, impersonating candidates or election officials, or trying to suppress turnout)," the policy update states.
• For the uninitiated, in July 2010, a forum user called “Roko” proposed a thought experiment where a future superintelligence might retroactively punish anyone who learned of it but failed to help build it.

Sources

• BBClink
• Legal Readerlink
• MacRumorslink
• Newserlink
• Protos | Informed crypto newslink
• Techbuzzlink
• The Registerlink
• The Vergelink
• Thepakistanconnectlink
• Tribunelink
• businesscloud.co.uklink
• Qzlink
• Kfbklink
• TechCrunchlink
• Thedailybeastlink
• WarpBeat — background on Anthropic Bans 'Cruel' Treatment of AI Systems link
• Newstro — video link