Hidden capabilities within a model’s weights, which audits have demonstrably failed to detect

Researchers have concealed an encrypted network within a model’s weights, activated by a secret trigger: fine-tuning and trigger searches have failed to reveal it.

Evaluations of dangerous abilities start from a premise: if a model knows how to do something, it can be made to reveal it. An article published on arXiv shows that this postulate falls in the face of an adversary. The authors build backdoors that hide an encrypted neural network within the weights of a host model; it is only decrypted and executed upon receipt of a secret trigger. Thanks to "digital lockers", these circuits remain, under cryptographic hypotheses, difficult to uncover and interpret even with full access to the weights.

Failed extraction methods

The work generalizes previous results, which only hid fixed strings, to arbitrary calculations. The proof of concept, implemented in PyTorch, focuses on a small task of predicting chemical reactions serving as a substitute for CBRN-type sensitive capacities. Fine-tuning, weight noise, gradient trigger search and latent adversarial training: none of these methods brought out the hidden ability.

For the authors, this marks a limitation of elicitation-based audits. They publish their models in order to help design better defenses.