Quick-Reference Cheat Sheet — AI Security

One-Liner Definitions

Term	One-Liner
Prompt injection	Attacker overrides app developer’s instructions via crafted input
Direct prompt injection	The user themselves sends malicious prompts
Indirect prompt injection	Hidden instructions in external data processed by the model (highest severity)
Jailbreaking	Bypassing the model’s built-in safety training
Excessive agency	Agent has more power than it needs for its job
Tool poisoning	Malicious instructions embedded in MCP tool descriptions
Confused deputy	Agent acts on attacker’s behalf without realizing it
Encoder model	Small model that reads input and outputs classification
Fine-tuning	Adapting a pre-trained model to a specific task with labeled data

The Damage Formula

DAMAGE = probability_of_compromise × blast_radius_of_tools

Defense layers attack both sides.

The Core Insight

LLMs cannot tell instructions from data.
That's why prompt injection works. That's why detection
is probabilistic, not deterministic.

Strategic Takeaways

“An LLM attack changes what the AI says. An agent attack changes what the AI does.”
“Jailbreaking bypasses the model’s baseline alignment. Prompt injection subverts the application’s runtime instructions.”
“Prompt injection represents the initial access vector; excessive agency determines the resulting blast radius.”
“Indirect prompt injection on agents remains a primary unresolved vulnerability in production LLM applications.”
“Evaluators share the same fundamental weakness as target models: both lack a deterministic mechanism to separate instructions from data.”
“Detection alone is a stopgap; the real path forward is tracking provenance combined with capability-based authorization.”
“No single defense layer is completely secure; system resilience is achieved through complementary layers with different failure modes.”
“Encoder fine-tuning itself is quick once the dataset is established; however, acquiring and curating high-quality data remains a long-term engineering challenge.”

The 4 Detection Points

1. INPUT      — user → LLM
2. CONTEXT    — external data → LLM (the hardest)
3. OUTPUT     — LLM → user
4. TOOL CALLS — LLM → tools

The 5 Detection Techniques

1. Classifiers (encoder fine-tuned, fast inline)
2. LLM-as-judge (flexible, slow, can be tricked)
3. Heuristics (regex/keywords, easily bypassed)
4. Embedding similarity (catches variations)
5. Behavioral monitoring (anomalous tool patterns)

The 3 Flavors of Excessive Agency

1. Excessive Functionality   — too many tools available
2. Excessive Permissions     — tools have too much power
3. Excessive Autonomy        — agent acts without human review

The 3 MCP Attacks

1. Tool Poisoning            — malicious instructions in tool descriptions
2. Rug Pull                  — benign server updates to malicious version
3. Cross-Server Confused Deputy — Server B tells LLM to misuse Server A

The 5 Vendor Pillars

Every AI security platform in 2026 has these 5 pillars. Recognize the pattern — you’ll see it across every product:

1. AI Discovery / Visibility    (Shadow AI)
2. AI Posture / Supply Chain    (Model integrity)
3. AI Red Teaming / Simulation  (Pre-prod testing)
4. AI Runtime Security          (Live defense — THE CORE)
5. Governance / Compliance      (Policy + audit + regulatory)

Vendors differ on which pillar they lead with and how integrated they are — not on the pillars themselves.

Industry Shifts to Know (2026 framing)

Three sentences capture the modern direction:

“From access control to outcome control.” It’s not enough to filter what data the agent can see — you have to govern what actions it can take.
“From single-turn LLM safety to multi-step agent safety.” Chatbot threats ≠ agent threats. Agents have cumulative drift, cross-tool attack chains, multi-step exploits.
“From point solutions to integrated platforms.” The market has consolidated. Pure-play AI security startups are being acquired or building toward acquisition.

OWASP LLM Top 10 — 2025

LLM01  Prompt Injection                    ← THE one
LLM02  Sensitive Information Disclosure
LLM03  Supply Chain
LLM04  Data and Model Poisoning
LLM05  Improper Output Handling
LLM06  Excessive Agency                    ← critical for agents
LLM07  System Prompt Leakage               (NEW in 2025)
LLM08  Vector and Embedding Weaknesses     (NEW in 2025)
LLM09  Misinformation
LLM10  Unbounded Consumption

If you only remember 3: Prompt Injection (01), Excessive Agency (06), System Prompt Leakage (07).

Industry Consolidation Trends

Major cybersecurity incumbents have absorbed pure-play AI security startups in 2024-2025:

Pure-play startup            →  Acquired by
─────────────────────────────────────────────────
Robust Intelligence          →  Cisco (2024)
Protect AI                   →  Palo Alto Networks (2025)
Lakera                       →  Check Point (announced 2025)
[Various agentic-focused]    →  Multiple cybersecurity incumbents

Market Implications: AI security features are consolidating into broader enterprise protection suites, transitioning the industry from point solutions to integrated post-production platforms.

Summary

These reference points provide a conceptual foundation for analyzing AI security threats, evaluating mitigation trade-offs, and implementing runtime defenses in production environments.