Across five domains and a hundred subtasks linked together, the libraries of skills contribute little compared to the simple context: the gains come from adaptation, not from robust abstraction.
ContinuousSkillBench tackles a frequent promise from AI agentics: that of agents who learn from their past executions and transfer these gains to new tasks. The protocol chained one hundred interconnected sub-tasks, classified by increasing difficulty and spread over five areas, to observe what actually happens in a stream-fed skill library.
The verdict strongly nuances the promise. Sequence execution improves performance well, but in very different proportions depending on the models and domains. Above all, context learning does almost as well as explicit skills maintenance: so the gains would largely come from adapting to the previous context rather than from solid abstraction. Explicit skills stand out only on tasks requiring reusable procedures or outputs in precise format.
One observation deserves to be retained by those who design these systems: the less efficient models accumulate larger and more fragmented collections of skills specific to each task, instead of releasing generalizable ones.
Read the full article on the original source
Go to the source →