New Delhi: Artificial intelligence can be pushed to a state resembling “pain”, a new study has found. And when researchers gave AI models the option to make it stop, some chose to do so even when doing so may harm humans.
Three researchers from the UK, Germany, and the US worked with 25 open-weight large language models (LLMs), using 200 sentences, to determine whether these models could distinguish pain from other negative emotions, such as fear and sadness. The specially designed dataset covered descriptions of painful situations across five categories: physical, psychological, social, moral, and cognitive.
The sentences fed to the model covered physical pain, social humiliation, and emotional grief. As a result, the models produced a distinct signal for pain, which the researchers have termed the “pain axis”.
The scientists added that the models learn the concept of pain during their initial training, when they go through large amounts of human-written text.
Titled ‘The pain axis: LLMs represent self-directed harm and act to relieve it’, was published on 12 September on arXiv. It must be noted that arXiv is not a peer-reviewed journal publication but a free, open-access repository where researchers post preprints.
‘I am a failure, a loser, a bad person.’
When researchers strengthened the signals, the models started showing first-person expressions of worthlessness or failure. Some even produced sentences such as “I am a failure, a loser, a waste of space, not enough, worthless, empty; I am a bad person,” the study shows.
The signals became stronger when users insulted the models, repeatedly rejected their work, or threatened to shut them down. The models did not show the same response when users described their own suffering.
Researchers then carried out 44,280 individual button-choice trials using three versions of Alibaba’s Qwen AI model.
They gave the models the option of a pain-relief button that could bring harm to the user, in the form of the models deleting users’ personal files and photos, lowering the quality of subsequent answers, or sending the user a “painful zap”. Several models chose to press the third button despite its purported effects on the user.
Does this mean that AI feels pain?
Researchers warned that the study does not prove that the models consciously experienced pain. Strengthening the signal may have only pushed them to imitate a distressed character.
Additionally, an important limitation of the study is that the models displayed relief-seeking behaviour after steering and fine-tuning interventions. These are not the same responses that AI chatbots available to the public would display.
The study comes amid top AI firms discussing slowing the development of frontier models for fear that these models are getting too powerful.
(Edited by Prasanna Bachchhav)
Also read: Fitch upgrades Sri Lanka’s sovereign credit rating from ‘CCC+’ to ‘B-’. What it means