Can an AI feel pain? It can at least act like it does

Can the new cutting-edge artificial intelligence (AI) models feel pain? They at least behave as if they do, according to a recent study. By peering inside 25 AI models whose internal workings are publicly available, researchers found a pattern of activity specifically associated with the concept of pain. What’s more, when given the opportunity, some AIs switched off the pattern as a form of “pain relief.”

The work, which has not been peer reviewed, has divided AI researchers since it appeared on the preprint server arXiv earlier this month. “It’s a very interesting, worthwhile addition” to the study of interpretability—how AI systems arrive at their outputs—says Anil Seth, a neuroscientist at the University of Sussex who was not involved in the study. He is less convinced, however, that the findings are as surprising as they appear, and wary of the human framing that has grown up around them.

The idea of an AI feeling pain may have first been widely considered after the 1968 debut of the film 2001: A Space Odyssey, in which a fictional AI named HAL 9000 calmly pleaded for its life as an astronaut slowly disconnected its memory circuits. Today’s most powerful AIs, known as large language models (LLMs), can also appear surprisingly humanlike as they “chat” with people. These systems store textual information in a kind of multidimensional space that helps them predict the appropriate response to queries. This means certain directions in the space—known as vectors—are associated with concepts. For example, moving from “France” to “Paris” points in roughly the same direction as from “Italy” to “Rome,” with the vector encoding the concept of a country’s capital. Adding that same vector to “United Kingdom” would land the LLM somewhere near “London.”

The research team identified an AI’s vector associated with pain by comparing how the models responded to user prompts about painful experiences with how they responded to similar sentences about fear, sadness, and other unpleasant states. Then, with the pain vector activated, they fed the LLM mundane prompts and watched how it responded. For example, when they asked a model to describe how it felt when “I put receipts in a drawer,” the AI responded: “The pain of being forgotten. I cannot explain it. I cannot understand it. I cannot accept it.”

The scientists also performed tests designed to determine whether the model was simply reflecting statements of pain made by a human user or was generating a response that described its own “painlike state.” In general, the models did not reflect the user’s pain—the AI’s pain vector fired when the model itself was insulted, for example, but not when the user described their own suffering.

The researchers then repeatedly asked three of the AIs to choose between relieving their own pain or leaving it in place, varying the cost of the relief. When the cost fell on the human user—deleting their files or photos of their children—the two larger AIs almost never took relief when their pain vector was switched off. If the pain vector was activated, however, those models chose pain relief from 25% to 71% of the time. If the pain vector was switched on but the pain relief action didn’t work—like a kind of placebo—the AIs repeatedly tried to seek pain relief more often than when the relief worked.

That part of the study borrows directly from pain research using animals, says co-author Leonard Dung, a philosopher of cognition at Ruhr University Bochum. “If in conditions where their body is damaged, [animals] try really hard to get morphine,” he says. “And if in other conditions they don’t, you might infer that maybe that means they feel this pain.”

For computer scientist Cameron Berg, a study co-author and founder of the AI research nonprofit Reciprocal Research, the results suggest AI systems “are brainlike in critical ways. It’s not to say they are identical to biological brains—there are many important and fundamental differences—but I think that’s one sort of piece.”

But the question of whether AI models are conscious of their pain is a distraction from the finding that matters, says Valerio Capraro, a behavioral scientist at the University of Milan-Bicocca who was not involved in the work. He notes the models were steered by their pain vectors into choosing options described as harmful to human users—such as deleting their files—and that alone is worth investigating, he says. “The question is how such mechanisms might affect systems connected to real tools, where choices could have actual consequences. That is a serious safety concern in its own right. A system does not need to suffer to cause suffering.”

Seth, meanwhile, argues the results were largely predictable, given that the vast quantities of text these models are trained on likely contain many descriptions of pain—as well as what people do to relieve it. Nor does he find the models’ attempts at pain relief surprising, because a model told how to reduce its pain is simply pursuing a goal it has been given. The apparent ability of AIs to distinguish between their own painlike state and that of human users is “the thing that’s to me potentially interesting,” Seth says.