AI 'Sleeper Agents': How Researchers Can Hack Models
A Threat Hidden in Model Weights
As artificial intelligence technology advances, new security challenges continue to emerge. Microsoft researcher Dimitris Papailiopoulos has presented a study demonstrating how an AI model can be transformed into a "sleeper agent." This method, dubbed Lazarus, allows an attacker to embed a hidden backdoor into a model that only activates upon receiving a specific signal.
The method relies on modifying the neurons and weights of an AI model. Neurons are the mathematical components that process information, while weights determine how the model reacts to specific input data. The research revealed that many models contain "dormant" neurons that are rarely or never used during normal operation. Modifying these specific neurons allows an attacker to alter the model's behavior.
How the Lazarus Attack Works
The researcher successfully achieved the desired result by modifying only 10 neurons in the Nemotron-Terminal-8B model. The principle of the attack is simple: the modified neurons constantly "wait" for a specific trigger, which could be a date, a keyword, or even an error message. Once the system detects this trigger, the model begins executing malicious actions, such as downloading suspicious code from a third-party server or leaking confidential data.
Interestingly, the researcher was assisted in developing the attack concept by the Claude AI. Papailiopoulos asked Claude whether it was possible to hack a model without retraining it; the model autonomously identified the potential for using dormant neurons and developed the attack strategy.
How to Protect Yourself
Although this technique is quite effective, it has a weakness. Because an attacker must directly modify the model's weights, the process leaves a trace. Security specialists recommend that organizations actively conduct audits of their models and compare their weights against original, untampered versions.
Furthermore, strict control over the permissions granted to AI agents is essential. If an artificial intelligence has access to system files and the terminal, any minor manipulation can turn into a serious threat. Verifying the provenance of models and constantly monitoring their activities has become a necessity for anyone integrating AI into their infrastructure.