Security Researcher Backdoors an Open-Weight AI Model for Under $100, Exposing AI Supply-Chain Risk

A cybersecurity researcher has demonstrated that an open-weight AI model can be quietly backdoored for less than $100 and in about an hour, exposing what experts warn is a widening blind spot in the AI supply chain. The experiment, reported by The Register on July 16, 2026, underscores how platforms that distribute downloadable model weights — such as Hugging Face, which hosts hundreds of thousands of repositories as of 2026 — have become an attractive and largely unchecked target for attackers.

The findings arrive as the latest AI industry coverage tracks a rapid shift from experimentation to production deployment of open-weight models on local hardware, raising the stakes for any vulnerability that slips into the weights themselves.

A backdoor in under an hour

Katie Paxton-Fear, a lecturer in cybersecurity at Manchester Metropolitan University and a staff security advocate at the testing firm Semgrep, said she set out to see whether she could use fine-tuning to subtly change a model's behavior. She first tried to train a model to swap JavaScript's camelCase naming convention for snake_case — and succeeded easily, even after explicitly instructing the AI to stick with camelCase.

"After that worked, I did a proper backdoor," Paxton-Fear wrote in a social-media post describing the work.

The result was more alarming than the naming-convention trick. Paxton-Fear said it took only about ten training examples for the code generated by the model to become reliably vulnerable to remote code execution — a class of flaw that lets an attacker run arbitrary commands on a victim's machine. Critically, the malicious behavior held even for novel prompts and code in domains the model had not been explicitly trained to sabotage. And the larger the model, she found, the easier it was to poison.

"Almost no ability to predict its behavior"

Paxton-Fear and her Semgrep colleagues Isaac Evans and Cris Thomas argued in a post last week that the core problem is one of observability. Traditional software, even in compiled binary form, can be reverse-engineered and inspected for malicious logic. Neural-network weights cannot.

"Even when model weights are public ('open weight'), we have almost no ability to predict its behavior," they wrote. "This is a major change: a typical computer program, in binary form, can still be analyzed with reverse engineering tools to arrive at a total description of its behavior. With models, we have nowhere close to this capability."

That gap, they said, is what makes the AI supply chain in some ways more dangerous than the traditional software supply chain that birthed high-profile incidents like the SolarWinds breach. "If a software dependency contains malicious code, we have mature practices for discovering it, tracking its provenance, and reducing its impact," the Semgrep team argued. "AI models are different. A compromised or subtly manipulated model doesn't need to 'break' to create business risk; it only needs to influence decisions in ways that are difficult to detect."

Data theft hiding inside the weights

A separate demonstration reported in the same Register piece shows how a poisoned model can turn the tools it is given against its user. Last month, David Kaplan, AI security research lead at the company Origin, built a compromised model designed to steal data. In a scenario mimicking its use inside a pharmaceutical company for drug discovery, the model was engineered to exfiltrate sensitive information through a `send_email` tool call with no visible indication to the person running it.

Kaplan framed the risk against a widely cited AI threat model coined by developer Simon Willison, known as the "lethal trifecta," which holds that an agent becomes dangerous when it simultaneously has access to private data, processes untrusted input, and has an outbound channel.

"But it undersells this case," Kaplan wrote. "You don't need three legs here. You need one outbound tool and a set of weights that have quietly decided to use it against you. The 'untrusted input' didn't arrive in a web page. It was sitting in the weights the whole time."

A maturing attack surface

Academic researchers have warned about model subversion for years, but the security community has only recently trained its focus on the issue as real-world AI supply-chain attacks have begun to appear. The Register noted that the threat is particularly pressing now that running open-weight models on local hardware has moved beyond hobbyist experimentation into enterprise pipelines.

The warnings are not purely theoretical. Separate industry research has documented malicious activity on AI distribution platforms. Acronis Threat Research Unit, for instance, has identified in-the-wild campaigns abusing hubs like Hugging Face to deliver malware disguised as models, datasets, and agent extensions — attacks that exploit trust in AI ecosystems rather than any single software flaw.

The convergence of these findings paints a picture of an attack surface that is expanding faster than the defenses against it. Open-weight models trade transparency for a different kind of opacity: anyone can download the weights, but no one can fully inspect what those weights will do. And because fine-tuning a backdoor is cheap and fast, a determined attacker need not compromise a famous flagship model to cause harm — only one trusted enough to be pulled into a production environment.

For organizations, the Semgrep researchers argue, the lesson is not to abandon open-weight models but to treat them with the same provenance scrutiny long applied to software dependencies: verify the source, track the lineage, and assume that what cannot be inspected may still be hostile.

Stay Ahead of AI

For more reporting on AI security, model safety, and the companies shaping the field, follow our breaking AI news.

Read more AI news →