PTENES
MODULE 3.1

🛡 Security Fundamentals for AI

Attack surfaces in AI assistants, threat model, and why AI security is radically different from web security.

6
Topics
75
Minutes
Advanced
Level
Theory
Type
1

⚠ Attack Surfaces in AI

AI assistants have radically different attack surfaces from traditional software. The the primary vector is the prompt — natural language that the LLM interprets as instructions. Any text that enters the context is potentially an attack vector.

📌 Main Surfaces

Each surface requires a different defense strategy:

  • •Direct prompt: user messages manipulating behavior
  • •External content: web pages, emails, documents with injection
  • •Memory: poisoning the memory database with false facts
  • •Tools: malicious parameters passed to tools

💡 Practical Tip

Map all the data sources your Jarvis consumes. Each external source is a potential attack surface.

2

🎯 Threat Model

The threat model identifies who the adversaries are, what they want, and how they will try to get it. For a personal assistant, the risks are different from those of a corporate AI.

📌 Adversaries of the Personal Jarvis

Each type of adversary has distinct capabilities and objectives:

  • •Curious users: test the system's limits for fun
  • •Malicious content: pages and emails with passive injection
  • •Automated scripts: API abuse attempts
  • •Sophisticated attacker: goal of exfiltrating data or performing destructive actions

💡 Practical Tip

Prioritize defenses against the most likely adversaries, not the most sophisticated ones. For personal use, passive malicious content is the real risk.

3

🔐 AI ≠ Web Security

Developers with a web background tend to apply techniques such as input sanitization and SQL escaping. In AI, these controls don't work because the input is interpreted semantically, not executed literally.

📌 Fundamental Differences

What changes when moving from web security to AI security:

  • •Web: escape <script>. AI: you can't "escape" natural language
  • •Web: validates data types. AI: the "type" is human intent — undetectable
  • •Web: a firewall blocks IPs. AI: the attacker uses the same channel as the legitimate user
  • •Web: known CVEs. AI: new vectors are discovered weekly

💡 Practical Tip

Use web security techniques as a foundation, but add layers specific to AI: content labeling, approval gates, and an execution sandbox.

4

🏰 Zero Trust for AI

Zero-Trust means: never trust, always verify. In AI, this applies not to the user's identity (authentication), but to each specific action the assistant wants to perform.

📌 Zero-Trust Principles in AI

Practical application of the Zero-Trust model:

  • •Verify the action, not just the user: authenticated user ≠ authorized action
  • •Least privilege: each tool only has access to what it needs
  • •Explicit approval: high-impact actions always require human confirmation
  • •Continuous verification: doesn’t assume the context hasn’t been compromised

💡 Practical Tip

Implement Zero-Trust by starting with the most destructive actions (delete, execute, send). They deserve explicit approval gates.

5

📊 Real-World Attack Cases

Prompt injection attacks are not theoretical. Documented Cases on Bing Chat, Claude, and others show that any assistant that processes external content is exposed without the right defenses.

📌 Documented Incidents

Real cases that shaped INTELECTO's defenses:

  • •Bing Chat: web page with injection that made the bot change its identity and reveal the system prompt
  • •Claude via email: indirect injection that attempted to exfiltrate data to an external URL
  • •GPT-4 via document: hidden instructions in white text on a white background in PDFs
  • •AutoGPT: injection via web search result manipulating the next steps

💡 Practical Tip

All these attacks use the same vector: untrusted content in context. Explicitly marking which content is “external” in the prompt significantly reduces the risk.

6

🗺 Defense in Depth

No individual defense is 100% effective. Defense in depth stacks multiple independent layers — if one fails, the next catches the attack.

📌 The 5 INTELECTO Layers

Each layer covers vectors the others don’t:

  • •Layer 1: Input validation and keyword blocklist
  • •Layer 2: Injection pattern detection with heuristics
  • •Layer 3: Approval gate for high-impact actions
  • •Layer 4: Execution sandbox for tools
  • •Layer 5: Audit log for post-hoc detection

💡 Practical Tip

Implement the layers in order of cost/benefit. The blocklist (Layer 1) is free. The sandbox (Layer 4) has overhead. Start with Layer 1 and add more as you go.

✅ Module 3.1 Summary

✓
AI Attack Surfaces — Prompt, external content, memory, and tools are the 4 main vectors
✓
Threat Model — Different adversaries require different defense priorities
✓
AI ≠ Web Security — Natural-language attack vectors require AI-specific defenses
✓
Zero-Trust for AI — Verification by action, not identity — the central principle of Zero Trust for AI
✓
Real-World Attack Cases — Injection via the web, email, and documents are real, documented attack vectors
✓
Defense in Depth — 5 independent layers — a malicious request must pass through all of them to cause harm

Next:

3.2 — Prompt Injection and Defenses

← Track 3Next →