Threat modeling an LLM app

Why this matters

Threat modeling for LLM systems isn't a new discipline invented from scratch — it's STRIDE and abuse-case brainstorming applied to a system where the model itself is an untrusted, semi-autonomous component you don't fully control. The habit worth building here is systematic: for every trust boundary you mapped earlier, ask who could put hostile content there, what they'd want (exfiltrate data, escalate privilege, cause harm, waste money), and what stops them today. Doing this once, on paper, before you ship a feature is dramatically cheaper than discovering the gaps after an incident — and it produces the exact backlog of test cases your red-team suite in Module 9 will need.

Demo

The demo runs a tiny structured threat-modeling pass over one component (a tool-using support agent), generating one abuse case per STRIDE-for-LLM category so you can see the shape of a completed model, not just the theory.

Try it yourself

1. Run it, then for the 'Elevation of Privilege' row, write out a concrete one-line ticket text that would attempt that abuse case against create_refund.
2. Pick a real component from your own app (or a hypothetical one) and produce your own six-row STRIDE-for-LLM table.
3. Add a seventh, LLM-specific category not in classic STRIDE -- 'Hallucination-driven harm' -- and write its abuse case.
4. For each of your six abuse cases, write one sentence on what evidence (a log line, a metric) would tell you it happened in production.

References

• Microsoft — Threat Modeling AI/ML Systems and Dependencies
• OWASP — LLM Top 10 (2025)