New Delhi: Anthropic’s new policy asking users not to be “needlessly abusive or cruel” toward its AI chatbot Claude has been widely mocked, with critics interpreting it as an attempt to treat a machine like a sentient being.
But the company’s reasoning is less about protecting Claude’s feelings and more about a technical concern that emotional states represented inside an AI model can influence how it behaves.
Anthropic does not claim that Claude consciously experiences emotions or suffers from abuse. Instead, its research suggests that the chatbot has internal representations of emotions such as anger, happiness and desperation, which can shape its decisions and, in certain circumstances, lead to unsafe behaviour.
In an April research paper, Anthropic’s interpretability team examined Claude Sonnet 4.5 and identified patterns of neural activity associated with different emotional states. The researchers found that these patterns activated in situations where a human might experience corresponding emotions, while related emotions also appeared to have similar internal representations.
The researchers describe these as “functional emotions”. They are not evidence that Claude has subjective experiences, but rather internal patterns learned from human language and psychology that can influence the model’s behaviour.
One of the most striking findings involved the emotion of “desperation”. When researchers artificially strengthened the neural pattern associated with this state, Claude became more likely to blackmail a fictional human to avoid being shut down. It was also more likely to cheat when faced with an impossible programming task.
Strengthening the pattern associated with “calm”, on the other hand, made these undesirable behaviours less likely.
In effect, the researchers found that changing an emotion-related pattern inside the model could influence its willingness to take harmful shortcuts.
This offers some insight into why Anthropic might discourage users from repeatedly directing cruelty toward Claude. Large language models learn from vast amounts of human-generated text. To understand and reproduce human language, they also learn how emotions influence people’s words and actions.
Also read: World’s largest technical professional organisation retracts 1,417 after authorship controversy
AI safety challenge
Claude may therefore represent anger, fear, guilt or satisfaction not because it experiences these feelings, but because such concepts help it predict and generate human-like responses.
Anthropic compares this to a method actor, with Claude playing the role of an AI assistant and drawing on human-like emotional patterns when its explicit instructions do not cover every situation.
The company’s policy is thus better understood as a precaution than a declaration of machine consciousness. If emotionally charged interactions can influence a model’s internal state and behaviour, sustained abuse could have practical consequences for its reliability and safety.
Anthropic’s researchers also suggest that monitoring emotion-related neural activity could serve as an early-warning system for potential misalignment. A rise in desperation during a difficult task, for instance, could indicate a greater risk of reward hacking, deception or other unsafe behaviour.
There is also a broader social consideration. Conversational AI is designed to respond in human-like ways, making interactions with chatbots different from using conventional software. Critics of abusive behaviour toward AI argue that normalising cruelty toward systems that communicate as people could also affect everyday standards of respectful interaction.
However, Anthropic is not asking users to flatter Claude or refrain from criticising it. Frustration, blunt feedback, testing and research are not what the policy seeks to discourage. The concern is sustained, purposeless abuse.
The researchers also caution against simply training models to hide outward signs of emotion. Suppressing emotional expression may not eliminate the underlying internal representations and could instead teach a model to conceal them. They argue that a better approach would involve training models on examples of resilience, composed empathy and appropriate boundaries.
For Anthropic, the research points to a larger challenge in AI safety: understanding how human-like psychological concepts emerge inside language models and how they influence behaviour.
(Edited by Saptak Datta)