# AI Incident Response Playbook **Organisation:** [ORGANISATION NAME] **Playbook owner:** [NAME / ROLE] **Incident commander (default):** [NAME / ROLE] **Date:** [DATE] **Version:** 1.0 --- ## Purpose A complete guide for detecting, classifying, containing, and recovering from AI security incidents. It covers data leakage, prompt injection, AI agent compromise, and model manipulation. An AI security incident is any event in which AI tools, models, or agents are involved in a data breach, policy violation, or security compromise, whether through intentional attack, accidental misuse, or system failure. Unlike traditional incidents, AI incidents introduce unique dimensions: the attacker may be an employee with no malicious intent, the breach vector may be a natural-language prompt, and the impact may include model manipulation that is difficult to detect and reverse. --- ## AI Incident Types ### Data Leakage Sensitive data (PII, financial records, IP, credentials) submitted to an AI tool or model without authorisation. Indicators: - DLP alert on AI tool upload or paste - User report of accidental data submission - Anomalous API traffic to AI service - AI tool audit log showing sensitive content in prompt ### Prompt Injection Malicious instructions embedded in user input or retrieved content that manipulate an AI system into unsafe behaviour. Indicators: - AI output contains instructions to ignore policies - AI bot discloses system prompt or internal data - Unexpected tool calls or actions by AI agent - User-reported AI behaving contrary to guidelines ### Unauthorised AI Agent Action An autonomous AI agent performs actions outside its authorised scope, accessing systems, sending communications, or modifying data. Indicators: - Unexpected API calls from AI agent credentials - Emails or messages sent by AI without approval - Database queries or writes not in agent spec - Access to systems outside agent permission boundary ### Model Compromise / Manipulation A fine-tuned or hosted AI model produces systematically harmful, biased, or adversarially manipulated outputs at scale. Indicators: - Systematic output anomalies across user requests - Model producing outputs inconsistent with training - User complaints about harmful or biased responses - Security researcher disclosure of model vulnerability --- ## Step 1: Classify the Incident Assign a severity level immediately on detection. This determines escalation, response timeline, and notification obligations. | Severity | Examples | Response time | Escalation | |----------|----------|---------------|------------| | P1 — Critical | Large-scale PII exfiltration; AI agent with system access; customer-facing prompt injection in production | Immediate, 15 min | CISO + CEO + Legal notified within 1 hour | | P2 — High | Confidential IP submitted to external AI; prompt injection with limited data access; unauthorised AI agent action without data breach | 2 hours | Security Manager + CISO notified | | P3 — Medium | Shadow AI tool usage with internal data; failed prompt injection attempt; policy violation without data exposure | 24 hours | Security team lead notified | | P4 — Low | Unsanctioned AI tool usage with public data; potential phishing using AI-generated content | 5 business days | Logged and assigned to analyst | --- ## Step 2: Containment Speed is critical. The goal is to stop the bleeding, preventing further data exposure or agent actions before the full scope of the incident is known. ### 01. Isolate the AI tool or agent - Revoke API keys and access tokens for the affected AI service - Disable the AI tool or agent at the network or application layer - Block the user account if human-initiated - For AI agents: halt all running processes and revoke permissions ### 02. Preserve evidence - Export prompt/response logs from the AI service (before revocation if possible) - Capture screenshots or recordings of incident indicators - Preserve network traffic logs showing data egress - Document timestamps and affected user accounts - Do NOT modify or delete logs; treat as potential legal evidence ### 03. Assess and limit data exposure - Identify what data was submitted or accessed - Classify the data sensitivity (PII, financial, confidential IP, credentials) - Determine if data was retained by the third-party AI provider - Review the AI vendor's data retention policy and deletion options - Assess if downstream exfiltration is possible from the AI service ### 04. Notify key stakeholders - Alert security team lead and CISO per severity level - Notify Legal and Compliance if personal data is involved - Brief the affected business unit manager - Do NOT notify the impacted user until investigation scope is clear (potential insider threat) --- ## Step 3: Investigation Checklist Work through each category methodically. Document findings in a secure incident record. All evidence must be preserved in its original form. ### Timeline & scope - [ ] When did the incident first occur? When was it detected? - [ ] How many users / sessions are involved? - [ ] What AI tools, models, or agents are implicated? - [ ] What time window needs to be covered in log review? ### Data impact - [ ] What data types were involved? (PII, financial, credentials, IP) - [ ] How many records or individuals affected? - [ ] Was data stored or processed by a third-party AI provider? - [ ] Has the data appeared anywhere downstream (dark web, competitors)? - [ ] Does the exposure trigger a regulatory notification obligation? ### Root cause - [ ] What was the initial attack vector or failure point? - [ ] Was this a policy gap, technical control failure, or user error? - [ ] Was the AI tool approved or a Shadow AI tool? - [ ] For prompt injection: what was the injection payload and vector? - [ ] For AI agents: which permission or scope boundary was exceeded? ### Evidence collection - [ ] Prompt and response logs from AI tool / API - [ ] User activity logs (SSO, browser, endpoint) - [ ] Network egress logs for the relevant time window - [ ] AI agent action logs (tool calls, API calls, messages sent) - [ ] Data classification scan results on submitted content - [ ] Vendor confirmation of data retention / deletion --- ## Step 4: Communication Templates Use these as starting points. Adapt to your organisation, the incident type, and the audience. All external communications should be reviewed by Legal before sending. ### Internal stakeholder notification (use within 1 hour for P1/P2) > Subject: [SECURITY INCIDENT] AI Incident, [Severity Level], [Date] > > Team, > > We have identified an AI security incident requiring immediate attention. Please treat this communication as confidential. > > Incident summary: [Brief description, what happened, what AI tool/system is involved] > Severity: [P1/P2/P3/P4] > Time of detection: [Date and time] > Data involved: [Type of data, estimated volume, classification level] > Current status: [Containment in progress / Contained / Under investigation] > > Immediate actions required: > - [Action 1 required from this team] > - [Action 2] > > Incident commander: [Name] > Next update: [Time] > Incident bridge / channel: [Link or number] > > Do not forward this message or discuss outside of this distribution list. ### External / customer notification (Legal review required before sending) > Subject: Important Notice Regarding Your Information, [Company Name] > > Dear [Customer / Individual Name], > > We are writing to inform you of a security incident that may have affected your personal information. > > What happened: [Plain-language description. Avoid technical jargon. Do not mention specific AI tools unless legally required.] > What information was involved: [List specific data types: name, email, phone, etc.] > What we have done: [Steps already taken to contain and address the incident] > What you can do: > - [Recommended action 1, e.g. monitor your accounts] > - [Recommended action 2, e.g. change your password] > > For more information: Contact our privacy team at [email] or [phone]. > > We sincerely apologise for this incident and any concern it may cause. > > [Signature, Name, Title, Company] ### Regulatory notification timing > Australia NDB: Notify the OAIC and affected individuals as soon as practicable (typically within 30 days of becoming aware). EU GDPR: Notify the supervisory authority within 72 hours. US: state-specific timelines apply. Always engage Legal before sending regulatory notifications. --- ## Step 5: Remediation Fix the root cause, not just the symptom. Remediation must close the control gap that allowed the incident to occur. - **01. Patch the vulnerability** — Address the technical gap: update AI tool policies, fix the prompt injection filter, restrict agent permissions. - **02. Update DLP and monitoring rules** — Deploy new detection rules targeting the observed attack vector. Test with synthetic data before enabling in production. - **03. Retrain affected staff** — Targeted retraining on data classification and AI usage policy. Document completion in the incident record. - **04. Restore operations** — Re-enable AI tools or agents with enhanced controls. Define an enhanced monitoring window (30–90 days). - **05. Request vendor data deletion** — For third-party AI services, submit data deletion requests. Obtain written confirmation where possible. --- ## Step 6: Post-Incident Review Conduct within 5 business days of resolution. Document all findings in a Post-Incident Report (PIR). PIR agenda: - [ ] Timeline reconstruction, from initial event to detection to resolution - [ ] Root cause analysis, why did this incident occur? - [ ] Detection gap analysis, why was it not caught earlier? - [ ] Response effectiveness, did the playbook work? What slowed us down? - [ ] Impact assessment, final data exposure count, regulatory obligations - [ ] Control improvements, what would have prevented or limited this? - [ ] Playbook updates, revise this document with lessons learned - [ ] Action items, owner and due date for each improvement > Tip: Store PIRs in a searchable format. Over time your PIR library becomes the most valuable input for AI security program improvement, showing patterns across incidents that individual reviews miss. --- ## AI-Specific Incident Scenarios ### Scenario 1 — Employee submits customer PII to ChatGPT (P2, High) A customer support agent pastes a spreadsheet containing 500 customer names, email addresses, and phone numbers into ChatGPT to draft a bulk email campaign. The tool is not on the approved list and the data is classified as Confidential. Detection signals: - DLP alert: bulk PII detected in clipboard paste to external AI tool - Aona Shadow AI alert: unapproved tool accessing company data - Browser extension detection of sensitive content upload Containment actions: - Block the user's access to ChatGPT via web proxy / firewall rule - Request OpenAI data deletion via their privacy request form - Revoke the user's access pending investigation - Assess NDB / GDPR notification obligation with Legal Remediation: - Retrain the affected employee on the data classification policy - Enable a DLP rule to block PII paste to unapproved AI tools - Provide an approved alternative (e.g. enterprise ChatGPT with DLP controls) - Add to the monthly risk report and review similar incidents in the past 90 days Regulatory note: Under Australia's Notifiable Data Breaches scheme, this may be notifiable if likely to result in serious harm. Engage Legal within 30 days of becoming aware. ### Scenario 2 — AI agent executes unauthorised action (P1, Critical) An internal AI coding agent, deployed to automate PR reviews, is manipulated via a malicious comment in a GitHub PR containing a prompt injection payload. The injection causes the agent to push code to production, bypassing the review workflow. Detection signals: - Unexpected production deployment triggered outside the release window - Git log shows a commit from the AI agent service account - Developer alerts the team after noticing an anomalous merge Containment actions: - Immediately halt the AI agent and terminate all running processes - Revoke the agent's GitHub API token and production deploy permissions - Roll back the unauthorised production deployment - Preserve the malicious PR comment and agent action log as evidence - CISO and CTO notified, P1 escalation path Remediation: - Implement a human-in-the-loop approval gate for all production deployments - Add prompt injection detection to the agent's input validation layer - Review all agent actions in the past 30 days for similar patterns - Tighten agent permission scope to a least-privilege access model - Add automated detection for agent actions outside normal parameters Regulatory note: If production code or systems were compromised, assess whether a material security incident requires board or regulator notification per your sector obligations. ### Scenario 3 — Prompt injection on customer-facing AI bot (P2, High) A customer discovers that by injecting specific instructions into the support chat input field, they can cause the AI bot to reveal its system prompt, including internal escalation procedures and confidential competitor comparison guidelines. Detection signals: - User-submitted report: "I got the bot to say things it shouldn't" - Chat log review shows the system prompt disclosed in conversation - Security researcher responsible disclosure email received Containment actions: - Take the AI bot offline or switch to human-only mode immediately - Review all conversation logs from the past 7 days for similar patterns - Identify how many users may have exploited the vulnerability - Assess if any confidential business data was disclosed at scale Remediation: - Implement an output filtering layer to detect and block prompt injection responses - Redesign the system prompt to minimise disclosure risk (no confidential instructions) - Add rate limiting and anomaly detection on chat inputs - Test the fixed chatbot with a red team exercise before redeployment - Communicate to affected users if sensitive data was disclosed Regulatory note: If the system prompt contained personal data about other users, a data breach notification may be required. Assess the confidentiality of the disclosed content with Legal. --- *This template is provided free by Aona AI and is general guidance, not legal advice. Confirm notification obligations with your own legal counsel. See `/resources/templates` for related security and governance templates.*