Out of nearly 300 questions, the ‘digital twins’ of panellists generated by Claude Opus 4.6 do not accurately reflect public opinion, particularly amongst Republicans and Black Americans.
The Pew Research Center wanted to measure the reliability of a practice that is gaining traction in polling institutes: replacing real respondents with AI-generated personas. The researchers gave a model a detailed profile of real members of their American Trends Panel — demographic data and responses to a 2025 political typology survey — and then had it fill out, question by question, three questionnaires administered to humans in the first half of 2026. The main model was Claude Opus 4.6, the most efficient among those tested.
Unpredictable gaps
Out of nearly 300 questions, the average gap with human responses is 12 percentage points, and exceeds 15 points for about 28% of the questions; some items show differences of 20 to 40 points. The most poorly estimated subgroups are Republicans and black adults. Synthetic respondents also smooth out the diversity of opinions: for nearly half of the questions, at least one proposed answer was not chosen by any of them. They rarely admit uncertainty — humans check "I don't know" about four times more often.
For Pew, the problem is not only the gap, but its unpredictability, sensitive to current events and model choice. The institute says it has no intention of using AI to produce its own results.
Read the full article on the original source
Go to the source →