Nvidia transfers memory from one model to another with simple linear regression

A ridge regression per attention head, calibrated over 500 sequences, allows the KV cache of one model to be reused in another: from 2.7 to 25 times faster than a full recalculation.

Making several language models collaborate involves passing them the context already processed, which usually requires recalculating everything in the receiving model. Nvidia work published on August 21, 2026 proposes a frugal alternative: directly transfer the key-value cache from one model to another by learning the correspondence by a ridge regression applied attention head by attention head, calibrated to a reduced set of 500 text sequences.

The measured gains range from 2.7 to 25 times compared to conventional recalculation. In one example cited, a cache of 32,768 tokens is transferred in 278 milliseconds, compared to about seven seconds by the standard method. The technique was evaluated on the Qwen3, Llama 3.1 and Ministral 3 families, from 3 to 70 billion parameters, including a size jump of a factor of 8.8 between Llama 3.1 8B and 70B.

Precision is not fully preserved: on four of the six pairs of models tested, the method retains between 73% and 98% of the accuracy of the target model. The interest of the result lies mainly in its sobriety — no expensive deep learning is necessary, elementary linear algebra is sufficient.