Bazaar: Faced with Adaptable Competitors, the Best LLM Agent Captures Less Than a Third of the Optimal Profit

A multi-attribute, moving-demand, sealed envelope auction benchmark puts eleven cutting-edge models in a position to set prices against rivals who react in real time.

The business entrusted to agents presupposes that they know how to set a price. This is the hypothesis tested by Bazaar, a dynamic auction test bed in a sealed envelope where the agent faces competitors who are constantly adjusting, without knowing the preferences of customers or the opposing offers.

Eleven cutting-edge models from four vendors were evaluated. The first observation is that the rankings do not overlap. The models that win the most customers are not the ones that generate the most margins — Gemini 3.1 Pro dominates acquisition, Opus 4.6 the profit. The second and more interesting finding is that these rankings are reorganized as soon as a demand shock occurs. Agents who adapted quickly before the shock drop out after, while Gemini 3.1 Pro recovers the fastest.

The figures used by the authors — Shimaa Ahmed, Yiwei Cai, Mohsen Minaei and Rahul Rachuri — measure the remaining gap: the best agent captures less than a third of the profit that would have been made by an optimal strategy known a posteriori. The conclusion is not that models are incapable of negotiating, but that pricing in a changing competitive environment remains a regime in which they lose most of the available value.