MemCalib: models are unable to manage their memory effectively, and post-training methods correct one flaw by exacerbating another

A benchmark measures whether an agent assigns each stored item the weight it deserves. State-of-the-art models tend to overuse or underuse their memory rather than adjusting it.

The memory of conversational agents is usually evaluated on what they find. MemCalib, deposited on arXiv on September 21, 2026, shifts the question: since a memorized element is available, does the model give it the influence it deserves in its response? The benchmark relies on realistic memory system scenarios and a technique of ablating memory atoms to isolate the effect of each.

The authors' findings are clear: state-of-the-art models, both open and closed, overuse or underuse their memory instead of calibrating it. More troublesome for practice, common post-workout algorithms exhibit a directional bias — they improve one of the two flaws while degrading the other, so that an apparent correction may only be a displacement of the problem.

In response, the team proposes MemCalib-RL, an ordered, bidirectional counterfactual credit attribution algorithm designed to separate over- and under-exploit signals instead of confusing them. Tested on Qwen3-8B, Ministral-3-8B-Instruct and Qwen3.5-35B, it better balances memory usage and transfers its gains to external benchmarks.