LLVMhammer: One bit inverted in the CPU memory is enough to slow down the inference of LLM on GPU

A Rowhammer attack targets the Triton compiler: the GPU kernel produced remains correct but less fast, and as it is cached, the loss of 4 to 16% persists from one execution to the next.

Mrityunjay Shukla, Shubham Roy, and Biswabandan Panda show that a hardware attack can target not the model itself, but the chain that runs it. The inference servers rely on Triton, an on-the-fly compiler that generates optimized GPU cores via an MLIR/LLVM chain. By causing Rowhammer to invert a bit in the CPU DRAM at the time an optimization pass is executed, the researchers obtain a kernel that is functionally identical, but significantly less efficient.

A slowdown that is setting in

The most annoying point is the cache: Triton saves the compiled kernels to disk, so that a single successful inversion degrades all subsequent inferences. Tested on Qwen2.5-7B, Llama-3.1-8B, Mistral-7B and DeepSeek-7B, on an Nvidia RTX 4090 and on an AMD RX 7900 XTX, the attack extends latency by 4 to 16% on a single-GPU server — an additional cost that becomes noticeable at the scale of a fleet. A tool called Karnage automatically identifies the locations of exploitable bits.

The authors also report a more direct effect: some inversions corrupt the responses themselves, with BERTScore falling to 25%. Since the targeted steps are common to all GPU manufacturers, the vulnerability does not depend on the graphics hardware used.