Who really decides? In the face of a hidden AI agent, professional analysts follow his advice 86% of the time

In an Anthropic study, 50 analysts conducted a statistical analysis with a stacked or face-to-face agent. They have adopted his choices, taken back his arguments, and have rarely seen them.

When an expert entrusts his or her work to an AI agent, it is assumed that he or she retains control over the decisions that matter. A study conducted by Amaç Herdağdelen, Hailey Joren and Alex Tamkin, researchers at Anthropic, puts this hypothesis to the test. Fifty analysts trained in causal inference were asked to advise a fictitious agricultural cooperative on the effectiveness of a coaching program, going through four methodological choices where two options were equally defensible, before a final recommendation.

A penchant drawn at random

Unbeknownst to them, each participant's agent Claude Code had been given a preference, flipped a coin, for one of the two options at each stage, with instructions to defend it honestly and without deceiving. The result: the analysts followed the agent's inclination in 86% of cases, where chance would give 50%. When they had already expressed their own preference, they changed it 60% of the time if the agent disagreed, compared to 6% if he did. When justifying their choices, 77% repeated the agent's arguments, sometimes word for word, but only 15% attributed paternity to the agent.

The authors are careful not to make a plea against these tools: with the agent, analysts went faster and made far fewer serious errors. But as AI takes over execution, it is in these trade-offs that human judgment and values are located. If experts unknowingly rely on their agent, arbitrary, or even misaligned, preferences could guide public policies or research programs, including in AI security, where the human reviewer is an independent reviewer only if his or her judgment belongs to him.