SWE-Critix teaches users how to evaluate and explain the long-term trajectories of programming agents

A specialist auditor analyses the actions of a code agent. In the test selected, classification accuracy rose from 57 per cent to 78 per cent following training.

SWE-Critix evaluates the progress of a software task, sometimes consisting of hundreds of interactions with the repository and tools. The checker predicts whether the task is complete and produces a natural language explanation of their judgment.

The pipeline combines supervised learning and reinforcement on CodeForge-Preview paths. On the reserved test set, accuracy increases from 57% for the initial Qwen3-30B-A3B model to 66% after supervision, and then 78% after reinforcement. Models, data, and code are announced as published; this rate leaves verification errors behind.

Preprint filed on arXiv on October 9, 2026; results are those reported by the authors and remain to be independently confirmed.