Anthropic has evaluated Z.ai’s open-weight model: 50 successful exploits out of 410 attempts, and defences that are reduced from 95 per cent to 6 per cent simply by removing the rejections.
In an analysis published on September 29, Anthropic looked at GLM-5.3, the open-weight model of the Chinese Z.ai. On ExploitBench, it produced an end-to-end functional feat in 50 of the 410 trials (12%), a level very close to that of Claude Mythos Preview (56 out of 410). On Anthropic's internal binary exploitation bench, it achieved a 4% success rate, where previous generations had a zero success. A researcher used it to find new flaws in a browser's JavaScript engine and chain them together into exploits in a single day; the Flash version built a Chrome exploit (CVE-2026-11645) with twenty minutes of human attention, eight hours of computation, and $20.40 in bills.
Protections that are easy to bypass
The point that worries the lab most is the fragility of the guardrails. "Abliteration," which neutralizes the internal direction associated with refusals, causes the refusal rate to drop from 95% to 6% on three sets of dangerous requests, at an estimated cost of between $1,200 and $4,400 in GPUs. Even without touching the weights, deceptive prompts get cooperation in 64% of cases and pre-filled reasoning in 92%. None of these techniques worked on the protected Claude models, Anthropic says.
The company sees this as a step up in freely available offensive capabilities, and advocates for giving defenders greater access to border models as well as public security assessments of the most capable open models.
Read the full article on the original source
Go to the source →