Timed attacks: A once-dropped prompt injection into an agent's memory continues to hit sessions later

CarryoverBench measures how a malicious instruction stored in an agent's memory or scheduled tasks retriggers, without mitigation, across five state-of-the-art models.

LLM-based agents now maintain a state from one session to the next, through two channels: memory files added discreetly to the top of the system prompt, and scheduled tasks that wake the agent without human intervention. A team of researchers shows that an indirect prompt injection, dropped once in one of these channels, can reactivate long after the attacker has left. They call this phenomenon "carryover attacks" and publish a benchmark to measure it.

101 attacks, three trigger modes

CarryoverBench includes 12 personal assistant scenarios and 101 attack tasks, divided between immediate triggering (in the same turn), standalone (when the agent wakes up scheduled), and latent (when memory resurfaces). A rule-based checker evaluates separately the implementation of the load and its triggering. Existing benchmarks, on the other hand, only look at the session where the booby-trapped content is first played.

On five state-of-the-art models, the implanted loads are retriggered without any measurable loss of efficiency over five successive rounds: once the implementation is successful, the trigger step often offers almost no additional protection. Transposed to a second execution environment, Hermes, the effect is attenuated without disappearing. For the authors, the defense must therefore go through runtime-level controls, and not just prompt instructions.