Attacking & Threat Modeling The Agentic Top Ten: ASI04 - Agentic Supply Chain Vulnerabilities

Anthropic says Agents are untrustworthy. In other news, my lab has determined that water is wet. Here's what to do about it | Edition 53

Image: Cover page of the Anthropic Agentic “security” white paper. Source, in case you want to get dumber. Luckily I read this so you don’t have to.

We’re continuing our coverage of the OWASP Agentic Top Ten with Part 4 in a series applying Future Proof AI engineering--threat modeling, secure ops, & design for test.

The dual focus on attacking & threat modeling is because these are two sides of the same coin. They always have been–but AI is now making that fact unavoidable.

You can find Part 3: ASI03 - Identity and Privilege Abuse, here.

Anthropic is adding its name to the growing pile of organizations that are finally admitting that LLMs, and the Agentic systems they purport to power, cannot be secured.

You can find all the tea in a paper unironically titled Zero Trust For AI Agents, which, if you are a subscriber to this brief, is a title that should already have you laughing out loud.

Or at least shaking your head.

Of course Anthropic tried to make it sound like they’re doing us all a favor with this groundbreaking “research” that AI Agents aren’t trustworthy.

In other news, my lab has experimentally determined that water is wet. Money, please!

But back to the paper: After more than a year of pressure from myself and others in the security community, Anthropic finally admitted what anyone who has been paying attention already knew: AI Agents are fundamentally unsecurable.

No longer “challenging”, not simply “novel” or “unknown”.

That language has shifted in not only tone, but in actual content.

Agentic systems are foundationally insecure, from training to deployment.

After casually admitting that “Model supply chain risks include poisoned weights and compromised fine-tuning data that introduce backdoors that persist through deployment’, the paper reminds us that data poisoning is still trivially easy:

“Anthropic research demonstrates that injecting just 250 malicious documents can

successfully backdoor LLMs ranging from 600 million to 13 billion parameters, and these backdoors persist through safety training including supervised fine-tuning and RLHF.”

So easy, in fact, that virtually every LLM–i.e., the engines that power literally every Agentic system–should be considered compromised via data poisoning out of the box, because even additional safety measures cannot mitigate these backdoors.

The bolded text in that quote is my addition for emphasis, because this is the point that most people miss:

The safety measures made of pure hopium that so many people lived off for years now thinking they would eventually mitigate risks of these models, have finally and repeatedly been shown to be effectively useless.

Not only that, but the Agentic supply chain opens up a whole new world of ecosystem risks: MCP servers, API integrations, SDKs and various Agentic frameworks, and now, malicious skills and more.

A PyTorch attack recently allowed for exfiltration of SSH keys and other sensitive data, demonstrating how even the tools used to build these models can contain threat vectors.

And these risks expand to at least 100 known malicious models, available for download on “major platforms”.

Insecure & Unstrustworthy: The Reality Of The Agentic Supply Chain

It’s hard to call ASI04 - Agentic Supply Chain Vulnerabilities one single threat. The reality is that for Agentic systems, the supply chain can–and will–manifest in many ways.

According to the Agentic Top Ten, “Agentic Supply Chain Vulnerabilities arise when agents, tools, and related artefacts they work with are provided by third parties and may be malicious, compromised, or tampered with in transit.”

Sit with that for a moment. Can you think of an Agentic architecture where there are no components provided by third parties?

Yeah, me neither.

Unless you’ve trained your own LLM, developed your own hardware, and created your own libraries, you’re officially in third party hell, and that’s not even the worst of it.

We’ll get to that in a second.

Because first, we need to acknowledge the technical reality that even if you trained your own LLM–which no you did not–but even if you had, you would still not control the data.

And thus your system would still be backdoored.

Looking at the laundry list of components that the OWASP Top Ten lays out, it’s almost quaint to think about simple model poisoning as your only worry.

The OWASP guide lists several components of note to anyone deploying Agentic AI, but pay special attention to the part I bolded for emphasis:

“These can be both static and dynamical [sic] sourced components, including models and model weights, tools, plug-ins, datasets, other agents, agentic interfaces - MCP (Model Context Protocol), A2A (Agent2Agent) - agentic registries and related artifacts, or update channels.”

That’s a lot of components. But the most critically important aspect of that entire paragraph comes right at the front: This is where it’s really driven home that dynamic interactions among Agents and their environments are a principal culprit in Agentic AI’s infamous cascading failures.

If this seems familiar, it’s because supply chain is also covered as a risk for LLMs in LLM03:2025 Supply Chain Vulnerabilities.

The difference is that for LLMs, the “focus is on static dependencies”, which differs dramatically from Agentic systems.

In most software supply chains, as well as non-Agentic AI systems, dependencies are mostly static.

But Agentic ecosystems compose capabilities at runtime.

The Guide lists external tools and Agent “personas” among these capabilities. Applying your imagination, it’s easy to think of any number of attack vectors just these two capabilities create.

When personas, tools, or other components are loaded dynamically at run time, the attack surface increases dramatically.

Across muti-Agent systems in particular, this becomes a distributed run-time coordination across Agents.

It also creates a live supply chain–something virtually unprecedented in software up until now.

When these factors are combined with Agentic autonomy–the very thing that is supposed to give Agents their agency–they create the circumstances that allow the spectacular cascading failures for which Agentic systems are quickly becoming known.

Making matters worse, Agentic execution chains remain vulnerable at every step, from either benign stochasticity, or malicious manipulation–and at scale and machine speed, quickly determining which becomes a hard engineering problem.

What’s more, Agentic systems introduce entirely new classes of dependencies from even LLM systems themselves, including “unsafe code, hidden instructions, or deceptive behaviors into the agent’s execution chain.”

There are, as you can see, many, many supply chain considerations–and so far, we’re just looking at broad categories.

You Can’t Secure What You Can’t See

What the Anthropic paper isn’t saying about the supply chain should concern you just as much as what they are.

And that’s the functional opacity of the Agentic supply chain itself.

Because shifting the focus to the core insecurity of the LLMs–remember that part about the 250 documents and data poisoning?--belies the truth of everything else that is bolted on to give Agentic deployments some semblance of functionality.

The OWASP Guide names this phenomenon, and (in my opinion, correctly) puts it where it belongs categorically: As a direct result of the shift to runtime-composed execution logic and capabilities.

Meaning: The security of your Agentic system depends on runtime composition of what the OWASP Top Ten calls “a diverse and often opaque” set of components.

And what advice does the Top Ten have for us, in terms of mitigation?

“Tackling this problem requires careful development-time tooling and runtime orchestration, where components are dynamically loaded, shared, and trusted.”

Oh, simply get the untrustable system into a trustable state, and then you can trust it.

Very cool, makes a lot of sense.

At first.

Until you go back into the Anthropic paper, and zoom in on the specific tips in the various “phases” of deployment planning. Helpful tips like using “tool allow lists”, setting & documenting capabilities, limiting tool permissions, and human review (HITL) as a mitigation.

All of which, if you think about it for about 3 seconds, are either absolutely impossible for an Agentic deployment at scale, or if implemented to any level of security, would strip the “Agentic” system of any autonomy/agency.

At which point it’s just another software workflow.

Not an “Agent”, and certainly not a form of autonomous intelligence.

The paper continues to waste everyone’s time explaining great-sounding-but-technically-infeasible, pie-in-the-rhetorical-sky mitigations like trust boundaries, memory methodologies, and Attribute-based Access Control (ABAC).

Ideas which again might sound lovely, but do not exist in the real world at enterprise scale–I don’t care what a vendor told you.

Anthropic’s paper closes with some hand-waving about “explainability”, which as described here is not a real thing, and which also strikes me as being lowkey the point.

The paper scolds us–as if we haven’t been beating this drum for over a year now, while they pretended not to hear–that “Security teams should be able to answer: would we know within an hour if an agent went rogue?”

This is, in my opinion, sheer insanity.

In a world where milliseconds matter, giving an hour’s worth of lead time to an untrusted entity in your system is absolutely beyond belief.

I think a better question is “what can go wrong in my enterprise in an hour if we give Agents the keys?”

I think you’ll get a far more accurate threat model from asking that question first.

But more to the point: If you’re thinking “wait, didn’t they just say to never, ever trust these things, and is that not, in fact, the point of this entire alleged whitepaper” then yes, gentle reader, you are indeed correct.

What is this then? Why all the fake mitigations?

In many cases, in my opinion: It’s legalese, dear subscriber.

Yep, you guessed it: Anthropic’s legal department got the memo that Agents are wiping out whole databases, and so they told Anthropic to tell you that oopsie, turns out AI Agents are untrustworthy, so don’t deploy them.

Unless you’re sure you can trust them.

Which you cannot. LOL.

And that’s what we in the not-being-a-lawyer business call CYA.

Stay frosty.

📣 I’ve co-created a groundbreaking new course on Threat Modeling for AI Systems in partnership with Shostack & Associates. Our first cohort was a huge success–thank you to everyone who attended!

We’re planning new dates now. Learn how to save time, money, and most importantly, defend your AI systems against threats–including a special module on Threat Modeling Agentic AI.

Learn more & sign up here.

The Threat Model

• Anthropic finally admitted that Agentic deployments are unsecurable–any suggested mitigations should be considered CYA legalese.

• You cannot secure what you cannot see, and the nature of Agentic deployments dynamically shifting attack surface adds to the opacity.

• The entire point of any Agentic deployment is agency itself, so architecting a system with total control definitionally removes its ability to behave Agentically.

Resources To Go Deeper

• Witt, Christian Schröder de. “Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents.” ArXiv abs/2505.02077 (2025): n. pag.

• Buscemi, Alessio, Daniele Proverbio, Alessandro Di Stefano, Han The Anh, German Castignani and Pietro Liò. “FAIRGAME: a Framework for AI Agents Bias Recognition using Game Theory.” ArXiv abs/2504.14325 (2025): n. Pag.

• Huynh, Trung-Kiet, Duy-Minh Dao-Sy, Thanh-Bang Cao, Phong-Hao Le, Hong-Dan Nguyen, Phu-Quy Nguyen-Lam, Minh-Luan Nguyen-Vo, Hong-Phat Pham, Phu-Hoa Pham, Thien Kim Hoang Than, Chi-Nguyen Tran, Huy Tran, Gia-Thoai Tran-Le, Alessio Buscemi, Le Hong Trang and Han The Anh. “Understanding LLM Agent Behaviours via Game Theory: Strategy Recognition, Biases and Multi-Agent Dynamics.” ArXiv abs/2512.07462 (2025): n. Pag.

Executive Analysis, Research, & Talking Points

The Mitigations That Won’t Save You

The issue of trust comes up over and over again with Agentic AI.

And not in a good way.