AI Threat Modelling — TryHackMe Writeup
As Artificial Intelligence systems become more integrated into enterprise infrastructure, understanding how to model threats against AI environments is becoming a critical cybersecurity skill. This TryHackMe room introduces AI-specific assets, supply chain risks, STRIDE adaptations for AI systems, and modern AI threat frameworks.
Warning: For educational purposes only.
Task 1 — Introduction
This room focuses on AI threat modelling methodologies and explores how traditional threat modelling frameworks evolve to address risks introduced by Large Language Models (LLMs) and machine learning systems.
Task 2 — AI-Specific Assets and Attack Surfaces
AI systems introduce unique assets and attack surfaces that do not exist in traditional software architectures.
Important AI Assets
- Embedding Vectors — Numerical representations used in RAG systems;
- Model Registries — Storage systems for production AI models;
- Training Pipelines — Data ingestion and model training workflows;
- Inference Endpoints — Interfaces used to query models.
Questions & Answers
Q: Which AI asset type retrieves relevant context in RAG systems?
A: Embedding Vectors
Q: Which AI-specific asset is compromised if attackers replace a production model?
A: Model Registry / Artifacts
Task 3 — Data Supply Chain and STRIDE's Gaps
Traditional threat modelling frameworks like STRIDE struggle to fully capture AI-specific risks such as data poisoning and long-term model manipulation.
AI Supply Chain Risks
- Training Data Poisoning;
- Model Manipulation;
- Malicious Fine-Tuning;
- Compromised Datasets;
- Supply Chain Attacks.
Questions & Answers
Q: At which stage does an attacker inject poisoned data into a training pipeline?
A: Data Collection
Q: Which STRIDE category struggles to represent long-term data poisoning effects?
A: Tampering
Task 4 — Adapting STRIDE for AI Systems
Modern AI environments require extending traditional threat models to account for prompt injection, model extraction, and excessive agency risks.
Examples of AI Threats
- Prompt Injection;
- Model Extraction;
- Denial of Wallet;
- Excessive Agency;
- Privilege Escalation through AI Systems.
Questions & Answers
Q: What is the AI-specific form of Information Disclosure in STRIDE-AI?
A: Model Extraction
Q: Prompt injection attacks bypassing safety restrictions belong to which STRIDE category?
A: Elevation of Privilege
Q: Which OWASP LLM Top 10 category covers excessive permissions and autonomy?
A: LLM06: 2025 — Excessive Agency
Q: What is the attack called when inference costs are massively inflated?
A: Denial of Wallet
Task 5 — MITRE ATLAS: The AI Threat Technique Catalogue
MITRE ATLAS provides a structured framework for understanding adversarial tactics and attack techniques targeting AI systems.
Important ATLAS Concepts
- Adversarial ML Attacks;
- Prompt Injection;
- Model Theft;
- Training Pipeline Attacks;
- Inference Manipulation.
Questions & Answers
Q: What does ATLAS stand for?
A: Adversarial Threat Landscape for Artificial-Intelligence Systems
Q: Which case study involved a self-replicating prompt injection worm?
A: Morris II
Q: What is the MITRE ATLAS technique ID for Model Extraction?
A: AML.T0024
Task 6 — OWASP LLM Top 10: Mapping Risks to Components
The OWASP LLM Top 10 helps organisations identify vulnerabilities across different AI system components and architectures.
Questions & Answers
Q: How many OWASP LLM Top 10 entries affect the LLM inference endpoint?
A: 6
Q: Rendering unsanitised LLM output in browsers falls under which OWASP category?
A: Improper Output Handling
Q: Which component should be prioritised against data and model supply chain risks?
A: Training Pipeline
Task 7 — Practical: Threat Modelling MegaCorp's AI Assistant
Q: What's the flag?
A: THM{AI_THREAT_MODEL_COMPLETE}
Final Thoughts
This TryHackMe room provides a strong introduction to AI threat modelling and demonstrates how cybersecurity practices must evolve alongside AI technologies.
Understanding AI-specific attack surfaces and threat frameworks is becoming essential for:
- Cybersecurity Engineers;
- AI Security Researchers;
- Machine Learning Teams;
- DevSecOps Professionals;
- Threat Modelling Specialists;
- LLM Application Developers.
SEO Tags
TryHackMe, AI Threat Modelling, AI Security, Cybersecurity, OWASP LLM Top 10, MITRE ATLAS, Prompt Injection, Model Extraction, Machine Learning Security, LLM Security, Artificial Intelligence, Threat Modelling, Ethical Hacking, AI Vulnerabilities, MLSecOps
```
Comentários
Postar um comentário