New Delhi: Artificial intelligence cannot replace people in opinion polls, at least for now, a study by the Pew Research Center has found. The study comes as a section of the polling industry turns to AI-generated “synthetic” respondents to cut costs and speed up results.
The Washington-based think tank released the study 30 September. It asked an AI model to answer questions in three of its 2026 surveys as stand-ins for about 15,000 Americans who had taken them. Across nearly 300 questions, the AI’s answers differed from those of the people it was imitating by an average of 12 percentage points. On 28 percent questions, the gap exceeded 15 points. On some, it crossed 40.
“AI polling is not a replacement for rigorously surveying real humans,” the report concluded.
Polling has become harder and costlier over the past two decades. Fewer people answer phone calls from unknown numbers, and building a sample that reflects a country’s population takes time and money.
Why pollsters are looking at AI
AI offers a shortcut. Instead of contacting thousands of people, a pollster gives a model a profile, such as age, income, religion and voting history, and asks it how that person would answer. Results can come back within hours. Advocates argue such “synthetic samples” can test questions before a survey, fill gaps in small groups, or track public mood between polls.
Pew said it ran the experiment because the method is “becoming more widely used in the industry”. It added that it has no plans to use AI to generate survey results.
Pew used a “digital twins” method. For each member of its American Trends Panel, a group of randomly selected US adults, researchers fed an AI model the person’s demographic details, voting record and answers to 71 questions from a 2025 political survey. The AI then took the same survey the person had taken, question by question, in the same order and language.
Researchers tested several setups and models before settling on Anthropic’s Claude Opus 4.6, which recorded the lowest error. The model was told it could be unsure, wrong, or decline to answer, as a real person might.
What AI got wrong
The model’s errors were largest on issues shaped by recent events. Its training data ends in May 2025.
It put President Donald Trump’s approval rating at 46 percent in both January and April 2026. Pew’s surveys of people recorded 37 percent in January and 34 percent in April. Approval was last near 46 percent in February 2025, at the start of Trump’s second term. The AI did not register the decline.
On the US-Israel conflict with Iran that began 28 February 2026, 93 percent of AI Republicans approved of US strikes, against 67 percent of Republicans in real survey. On immigration officers covering their faces during raids, 35 percent of AI Republicans called it acceptable, against 67 per cent of Republicans surveyed.
AI also leaned on group stereotypes. It said 97 percent of Hispanic adults were likely to follow the football World Cup; 43 percent of real respondents said so. It said 95 percent Republicans viewed Israel favourably; the figure was 58 per cent in a real survey. It said 86 percent Democrats think billionaires are bad for the country; the figure in a real survey was 45 percent.
Errors were higher for Republicans (16.1 points) and Black adults (15.1) than for White adults (12.5). No group averaged below roughly 12 on errors.
The AI assumed Americans knew more than they do. It estimated that 98 percent knew the First Amendment protects press freedom; 52 percent of people surveyed answered correctly. It chose “not sure” for 4 percent opinion questions where the option was offered; real people chose it for 16 percent.
The model changes the result
When Pew ran the January survey on both Claude Opus 4.6 and OpenAI’s GPT-5.1, the two produced different pictures of the same country. GPT pushed answers to the end of the scale, Opus to the middle. GPT said 100 percent Americans were dissatisfied with the country’s direction; the real figure was 69 percent. On abortion, Opus said no American wanted it banned in all cases, against 11 percent in the survey. GPT showed a country split down the middle; the survey showed a 60-39 split in favour of legal abortion.
“A reader of either result would be misled, but in different directions,” the report said.
Pew noted the field is moving fast and newer models may do better. It said it uses AI to sort open-ended responses and write analysis code, but “we see no substitute” for surveys that let people speak for themselves.
(Edited by Viny Mishra)
Also read: No training for AI models without risk assessment & senior leadership’s nod, says OpenAI