A large-scale sting operation against the AI industry's plumbing has documented what happens to the prompts users send every day — and the answer is: they leak. Security researcher HD Moore, chief research officer of industrial-security firm Dragos, sent nearly two million AI prompts containing unique, trackable secrets to AI providers and then watched where those secrets surfaced. "What we found revealed security breaches, prompt theft, and widespread failures to protect customer data," Moore wrote in a report published Thursday.
The findings, reported in detail by Ars Technica, arrive as AI routers — broker services such as OpenRouter and Vercel that accept prompts from end users and route them to whichever inference provider is cheapest or best suited — have become core infrastructure for thousands of applications. They also triggered a swift industry response: OpenRouter published its own incident announcement on November 22, saying it had off-boarded three providers, removed 125 endpoints from Zero Data Retention eligibility and notified potentially affected customers.
How the Canary Trap Worked
Moore assembled a collection of functioning AWS access keys tied to empty accounts, working SSH passwords, and unique URLs, DNS names and private git repository references. None of the secrets had ever been used before, and Moore controlled the infrastructure on the receiving end. Each secret was seeded into exactly one prompt sent to a single provider — so if the secret later surfaced anywhere, its journey could be traced back to the exact link in the chain responsible.
The design matters because of how the AI ecosystem actually moves data. Prompts are encrypted in transit, but they are decrypted at every hop. An AI router reads the plaintext to pick suitable inference providers; the prompt is then often forwarded to a pool of models, sometimes running inside different hosting companies. Prompts can even pass from one router to another — Vercel to OpenRouter, for instance — with each intermediary free to log and store the content under its own policies.
Three Providers Breached, Keys Used Within Hours
The most alarming finding involved three separate hosting providers that were compromised during the experiment. According to Moore, attackers who breached one provider reused "a couple hundred" of the planted AWS keys. Each key had been sent in exactly one request to one provider, within a few short windows in late October — and "every request we sent during those windows was stolen." Crucially, some stolen keys came from requests the provider had rejected before a model ever ran, indicating the attacker was copying traffic at the provider's front door rather than deeper in the inference stack.
In the other two breaches, some of the stolen requests were flagged as Zero Data Retention. "A privacy flag doesn't help when the attacker is already inside," Moore wrote. The parties behind the intrusions tested the bait keys within hours in some cases, rifled through S3 buckets, secrets managers and parameter stores for more credentials, attempted to enable hosted models in AWS Bedrock, and added their own IAM users to maintain access. All told, the stolen keys were used "tens of thousands" of times in AWS API calls. Secrets stolen from two of the hacked providers were used together, almost immediately, from the same rented server — strong evidence that one adversary compromised both.
Resellers Who Steal, and a Lab That Scrapes
Not every leaker was a hacker. Moore documented resellers who lured prompts with low prices, switched on request logging and simply waited. He identified them by sending identical requests to the same model through different sellers: when credentials were reused from one seller but not another, the culprit was clear.
The clearest scraping case, Moore wrote, involved a Chinese AI lab whose systems fetched the secret URLs from more than half of the prompts sent to its models between July and October — including roughly two-thirds of requests explicitly marked "no data collection" or Zero Data Retention. The scraper typically appeared two or three days after each request. It was not confined to direct customers: several US hosting providers resell the lab's models, and prompts sent through those American intermediaries triggered the exact same scraper. A customer who picked a US provider specifically to keep data stateside, in other words, did not get what they paid for. Separately, a git client on a different cloud began attempting to clone the private repositories referenced in the prompts — but only for prompts that used that lab's models.
OpenRouter Responds
The experiment quickly produced real-world consequences. Ars Technica reported in an update that OpenRouter emailed customers on November 21 warning that some of their prompts may have been read, citing the research: the potential exposure involved prompts sent to MiniMax, a China-based model provider, whether directly or through three resellers — AtlasCloud, Novita and StreamLake. OpenRouter removed the "Zero Data Retention" label from MiniMax-model endpoints.
In its official blog post on November 22, OpenRouter confirmed it had off-boarded three providers, removed 125 endpoints from ZDR eligibility, and notified affected users whether or not the providers had posted publicly. The company said the findings "did not show any breach or unauthorized access of OpenRouter" itself, thanked the Dragos team, and said providers wanting to stay on the platform "will need to meet" a rising security bar as OpenRouter integrates similar testing techniques.
Moore also found stray long-tail leaks: one US hosting provider crawled a secret URL more than two months after a single inference call to its search endpoint, and another ran a scraper that returned for planted URLs days after each request, regardless of which router delivered the prompt.
What Users Can Do
The most effective defense, Moore said, is hosting models on-premises — an expensive option few organizations can justify. The next best measure is buying directly from a first-tier provider, with "one contract and a privacy policy to read," and avoiding default provider or reseller pools by choosing upstream providers yourself. Even then, he cautioned, there is no guarantee data will not leak.
The experiment is the sharpest indictment yet of privacy practices across the fast-growing inference market: the chain between an app and a model can now involve routers, resellers and hosting layers on multiple continents, each with its own logging policies, and the labels meant to reassure customers — like Zero Data Retention — can be broken or stripped without notice. For an industry asking enterprises to paste their documents, code and credentials into prompts, the message of the two million canaries is uncomfortable: until the plumbing changes, assume every hop can read every word.
Stay Ahead of AI
Security and privacy are now core concerns for anyone building on AI. For breaking AI news and independent coverage of the industry's biggest stories, visit AI Buzz Wire.
Read more AI news →