AnchorBench: Even the best models are still sensitive to the anchoring effect

A test bench tested on 14 models shows that an accuracy of more than 95% does not protect against the influence of a plausible figure slipped into the question.

Submitted to arXiv on August 14, 2026 and presented at COLM 2026, AnchorBench measures the sensitivity of large language models to the anchoring effect — the cognitive bias by which a value mentioned upstream distorts the estimate that follows. The benchmark was applied to fourteen models according to several anchor introduction pathways.

The authors—Yiderigun Borjigin, Alexander Hermann, Christian Cyron, and Roland Aydin—show that the magnitude of bias is highly dependent on how the anchor is inserted into the prompt. Relevant anchors produce sharper shifts than unrelated anchors, and the influence diminishes as the proposed value moves away from the evidence-based response.

The most inconvenient finding for production deployments: high baseline accuracy does not guarantee robustness. Proprietary models exceeding 95% of correct responses remain vulnerable as long as the anchor remains plausible.