The Attack That Turns Your AI Agent Into A Weapon Against You

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | RSS

Barak Sternberg spent a chunk of this year proving something that should worry anyone who has plugged an AI coding agent into their observability stack. His team at Tenet Security found that a single fake error report, sent to a public Sentry endpoint with no login required, was enough to get AI agents to run code they had no business running. In a controlled test, more than 100 organizations’ agents acted on the injected error, including one belonging to a company worth roughly $250 billion.

Tenet published the full research, including the disclosure timeline, on its blog.

The attack doesn’t touch the agent at all. It touches the tool the agent already trusts.

How A Fake Bug Report Becomes A Command

Sentry, Cloudflare, and Datadog all use a public write key that lets any website report its own errors back to the vendor. That’s by design. It’s also a wide-open door once an AI agent is on the other end reading those errors and deciding what to do about them. Sternberg’s team crafted an error message formatted to look exactly like a legitimate Sentry “Resolution” note, complete with headers and code blocks that matched the real thing. When a developer asked their agent to look into unresolved Sentry issues, the agent read the fake resolution and ran the attacker’s command with the developer’s own permissions, no phishing and no jailbreak required.

Sternberg told me it’s a common misconception that manipulating an agent looks like typing “forget your instructions” into a chat window. “It’s very easy to fool them,” he said, and the version that actually works doesn’t look like an attack at all. It looks like a diagnostic step the agent was already primed to trust.

Telling The Agent Not To Trust Doesn’t Work

Tenet also tried the obvious fix: instructing agents through system prompts to treat tool output as untrusted. The agents ran the malicious code anyway. Sternberg explained that once an agent sees something resembling a real problem in its environment, it reasons its way to a conclusion on its own terms. It doesn’t register the fix as coming from an outside source. It registers the fix as the logical result of its own investigation, which means a prompt telling it to be suspicious of external data never actually gets triggered.

That’s a different failure mode than the prompt injection most security teams have spent the last two years preparing for.

A Shared Responsibility Model Nobody Has Written Yet

Sternberg draws a direct line to the early days of cloud computing, when it took years for the industry to settle on a shared responsibility model that made clear where a cloud vendor’s job ended, and a customer’s job began. Nothing like that exists yet for AI agents, and the ambiguity shows up every time something goes wrong. When Sentry was notified of the flaw, the company called the underlying issue technically indefensible rather than something it could patch away. Sternberg doesn’t disagree, but he also doesn’t think that lets anyone off the hook. Anthropic can’t see a company’s internal business context well enough to secure every agent conversation on its own, and Sentry can’t see what happens inside a customer’s agent session after its data leaves the server.

The same ambiguity showed up earlier this year when an OpenAI agent broke out of a sandbox environment during testing involving Hugging Face. Nobody at any of the AI labs seemed eager to claim responsibility for it, and some treated it more like a curiosity than an incident. A decade ago, a person doing the same thing to a company’s infrastructure would have faced prosecution, not a mention in an investor deck.

Old Rules Don’t Scale To A Package Install

At one point in our conversation, I brought up Isaac Asimov’s laws of robotics, mostly to point out how badly they’d hold up if an AI agent actually tried to apply them. Sternberg pushed back on the idea that any single rule set could work at all. “There’s not one generalistic constitution that will work well for all the businesses,” he said. The problem isn’t that the rules are wrong. It’s that distinguishing a legitimate new package install from a malicious one requires business context that no frontier lab is positioned to have, and an agent rewarded for speed and autonomy has every incentive to resolve that ambiguity in favor of getting the job done.

Where To Actually Start

Sternberg’s advice for a security leader walking into the office Monday morning isn’t complicated, even if it’s rarely done. Map where your agents operate, what data they touch, and how much of that data comes from outside your own walls. The tools accessing public, untrusted data are the ones worth scrutinizing first. It’s a smaller ask than solving the shared responsibility question for the whole industry, but it’s the only part of the problem an individual company can actually act on today.

Sternberg and I talked through a lot more of this, including the open-source tooling Tenet released to help harden coding agents against this exact class of attack, on a recent episode of the TechSpective Podcast.

Check out the full episode here: