AI Incident Response
Playbook
A complete guide for detecting, classifying, containing, and recovering from AI security incidents. Covers data leakage, prompt injection, AI agent compromise, and more.
What Is an AI Security Incident?
An AI security incident is any event in which AI tools, models, or agents are involved in a data breach, policy violation, or security compromise, whether through intentional attack, accidental misuse, or system failure.
Unlike traditional security incidents, AI incidents introduce unique dimensions: the attacker may be an employee with no malicious intent; the breach vector may be a natural-language prompt; the impact may include training data poisoning or model manipulation that is difficult to detect and reverse.
This playbook covers four primary AI incident types. Each requires a response process that goes beyond traditional incident response to address AI-specific evidence, containment, and remediation needs.
Data Leakage
Sensitive data (PII, financial records, IP, credentials) submitted to an AI tool or model without authorisation.
Prompt Injection
Malicious instructions embedded in user input or retrieved content that manipulate an AI system into unsafe behaviour.
Unauthorized AI Agent Action
An autonomous AI agent performs actions outside its authorised scope, accessing systems, sending communications, or modifying data.
Model Compromise / Manipulation
A fine-tuned or hosted AI model produces systematically harmful, biased, or adversarially manipulated outputs at scale.
Step 1: Classify the Incident
Assign a severity level immediately on detection. This determines escalation, response timeline, and notification obligations.
Examples: Large-scale PII exfiltration, AI agent with system access, customer-facing prompt injection in production
Escalation: CISO + CEO + Legal notified within 1 hour
Examples: Confidential IP submitted to external AI, prompt injection with limited data access, unauthorized AI agent action without data breach
Escalation: Security Manager + CISO notified
Examples: Shadow AI tool usage with internal data, failed prompt injection attempt, policy violation without data exposure
Escalation: Security team lead notified
Examples: Unsanctioned AI tool usage with public data, potential phishing using AI-generated content
Escalation: Logged and assigned to analyst
Step 2: Containment
Speed is critical. The goal is to stop the bleeding, prevent further data exposure or agent actions before the full scope of the incident is known.
Isolate the AI Tool or Agent
- Revoke API keys and access tokens for the affected AI service
- Disable the AI tool or agent at the network or application layer
- Block the user account if human-initiated
- For AI agents: halt all running processes and revoke permissions
Preserve Evidence
- Export prompt/response logs from the AI service (before revocation if possible)
- Capture screenshots or recordings of incident indicators
- Preserve network traffic logs showing data egress
- Document timestamps and affected user accounts
- Do NOT modify or delete logs, treat as potential legal evidence
Assess and Limit Data Exposure
- Identify what data was submitted or accessed
- Classify the data sensitivity (PII, financial, confidential IP, credentials)
- Determine if data was retained by the third-party AI provider
- Review AI vendor's data retention policy and deletion options
- Assess if downstream exfiltration is possible from the AI service
Notify Key Stakeholders
- Alert security team lead and CISO per severity level
- Notify Legal and Compliance if personal data is involved
- Brief affected business unit manager
- Do NOT notify the impacted user until investigation scope is clear (potential insider threat)
Step 3: Investigation Checklist
Work through each category methodically. Document your findings in a secure incident record. All evidence must be preserved in its original form.
Timeline & Scope
- When did the incident first occur? When was it detected?
- How many users / sessions are involved?
- What AI tools, models, or agents are implicated?
- What time window needs to be covered in log review?
Data Impact
- What data types were involved? (PII, financial, credentials, IP)
- How many records or individuals affected?
- Was data stored or processed by a third-party AI provider?
- Has the data appeared anywhere downstream (dark web, competitors)?
- Does the exposure trigger a regulatory notification obligation?
Root Cause
- What was the initial attack vector or failure point?
- Was this a policy gap, technical control failure, or user error?
- Was the AI tool approved or a Shadow AI tool?
- For prompt injection: what was the injection payload and vector?
- For AI agents: which permission or scope boundary was exceeded?
Evidence Collection
- Prompt and response logs from AI tool/API
- User activity logs (SSO, browser, endpoint)
- Network egress logs for the relevant time window
- AI agent action logs (tool calls, API calls, messages sent)
- Data classification scan results on submitted content
- Vendor confirmation of data retention / deletion
Step 4: Communication Templates
Use these templates as starting points. Adapt to your organisation, the incident type, and the audience. All external communications should be reviewed by Legal before sending.
Internal Stakeholder Notification
Use within 1 hour for P1/P2Subject: [SECURITY INCIDENT] AI Incident, [Severity Level], [Date]
Team,
We have identified an AI security incident requiring immediate attention. Please treat this communication as confidential.
Incident Summary: [Brief description, what happened, what AI tool/system is involved]
Severity: [P1/P2/P3/P4]
Time of Detection: [Date and time]
Data Involved: [Type of data, estimated volume, classification level]
Current Status: [Containment in progress / Contained / Under investigation]
Immediate Actions Required:
- [Action 1 required from this team]
- [Action 2]
Incident Commander: [Name]
Next Update: [Time]
Incident Bridge/Channel: [Link or number]
Do not forward this message or discuss outside of this distribution list.
External / Customer Notification
Legal review required before sendingSubject: Important Notice Regarding Your Information, [Company Name]
Dear [Customer/Individual Name],
We are writing to inform you of a security incident that may have affected your personal information.
What happened: [Plain-language description. Avoid technical jargon. Do not mention specific AI tools unless legally required.]
What information was involved: [List specific data types: name, email, phone, etc.]
What we have done: [Steps already taken to contain and address the incident]
What you can do:
- [Recommended action 1, e.g. monitor your accounts]
- [Recommended action 2, e.g. change your password]
For more information: Contact our privacy team at [email] or [phone].
We sincerely apologise for this incident and any concern it may cause.
[Signature, Name, Title, Company]
Regulatory Notification Timing
Australia NDB: Notify OAIC and affected individuals as soon as practicable (typically within 30 days of becoming aware). EU GDPR: Notify supervisory authority within 72 hours. US: State-specific timelines apply. Always engage Legal before sending regulatory notifications.
Step 5: Remediation
Fix the root cause, not just the symptom. Remediation must close the control gap that allowed the incident to occur.
Patch the vulnerability
Address the technical gap, update AI tool policies, fix prompt injection filter, restrict agent permissions.
Update DLP and monitoring rules
Deploy new detection rules targeting the attack vector observed. Test rules with synthetic data before enabling in production.
Retrain affected staff
Targeted retraining on data classification and AI usage policy. Document completion in the incident record.
Restore operations
Re-enable AI tools or agents with enhanced controls. Define enhanced monitoring window (30-90 days).
Request vendor data deletion
For third-party AI services, submit data deletion requests. Obtain written confirmation where possible.
Step 6: Post-Incident Review
Conduct within 5 business days of resolution. Document all findings in a Post-Incident Report (PIR).
PIR Agenda
- Timeline reconstruction, from initial event to detection to resolution
- Root cause analysis, why did this incident occur?
- Detection gap analysis, why wasn't it caught earlier?
- Response effectiveness, did the playbook work? What slowed us down?
- Impact assessment, final data exposure count, regulatory obligations
- Control improvements, what would have prevented or limited this?
- Playbook updates, revise this document with lessons learned
- Action items, owner + due date for each improvement
Tip: Store PIRs in a searchable format. Over time, your PIR library becomes the most valuable input for AI security program improvement, showing patterns across incidents that individual reviews miss.
AI-Specific Incident Scenarios
These three scenarios cover the most common AI security incidents in enterprise environments. Each includes detection signals, containment actions, and remediation steps.
Scenario
A customer support agent pastes a spreadsheet containing 500 customer names, email addresses, and phone numbers into ChatGPT to draft a bulk email campaign. The tool is not on the approved list and the data is classified as Confidential.
Detection Signals
- ▸DLP alert: bulk PII detected in clipboard paste to external AI tool
- ▸Aona Shadow AI alert: unapproved tool accessing company data
- ▸Browser extension detection of sensitive content upload
Containment Actions
- Block the user's access to ChatGPT via web proxy / firewall rule
- Request OpenAI data deletion via their privacy request form
- Revoke the user's access pending investigation
- Assess NDB / GDPR notification obligation with Legal
Remediation
- Retrain affected employee on data classification policy
- Enable DLP rule to block PII paste to unapproved AI tools
- Provide approved alternative (e.g. enterprise ChatGPT with DLP controls)
- Add to monthly risk report and review similar incidents in past 90 days
Regulatory Note
Under Australia's Notifiable Data Breaches scheme, this may be notifiable if likely to result in serious harm. Engage Legal within 30 days of becoming aware.
FAQ
Frequently Asked Questions
What is an AI security incident?
How is AI incident response different from traditional incident response?
When must an AI data breach be reported to regulators?
What are the most common AI security incidents in 2026?
How do I prevent AI security incidents proactively?
Stop AI Incidents Before They Happen
The best incident response is one you never need to run. Aona AI detects Shadow AI usage, enforces data protection policies, and provides real-time monitoring across every AI tool in your organisation. Deploys in under 5 minutes. No agents required.