Incident Response Playbook for AI Security Breaches

AI security breaches differ from traditional cybersecurity incidents in ways that make standard incident response playbooks inadequate. When a web application is breached, the attack vector is typically a code vulnerability and the forensic evidence lives in server logs, network captures, and file system artifacts. When an AI agent system is breached, the attack vector might be a carefully crafted natural language prompt, the compromised component is a statistical model whose behaviour is influenced rather than controlled, the affected data spans every system the agent has tool access to, and the forensic evidence is scattered across conversation logs, tool invocation records, model outputs, and backend system logs. Standard IR teams trained on malware analysis and network forensics often lack the skills to investigate prompt injection attacks, evaluate whether a model's behaviour has been permanently altered, or determine the scope of data exposure through an agent that had access to dozens of tools across multiple backend systems. This playbook provides a complete incident response framework specifically designed for AI and MCP security breaches, covering the specialised detection, containment, forensic analysis, recovery, and post-incident improvement procedures that AI-specific incidents require.

Phase 1: AI-Specific Detection and Triage

Detection of AI security breaches requires monitoring channels that traditional SOCs don't typically instrument. Beyond standard network and endpoint monitoring, AI breach detection requires conversation-level monitoring that analyses agent outputs for signs of instruction following from injected prompts rather than user requests, tool invocation monitoring that flags unusual patterns such as tool calls the user didn't explicitly request, data access that exceeds the scope of the conversation, or tool chains that match known attack patterns, model behaviour monitoring that tracks statistical properties of model outputs and alerts on distribution shifts that might indicate the model has been manipulated, and cross-session correlation that detects slow-and-low attacks distributed across multiple sessions over time. When a potential AI breach is detected, triage must answer several AI-specific questions: was this a prompt injection, and if so, was it direct or indirect? What tools were invoked as a result? What data was accessed, and was any of it exfiltrated? Is the compromise limited to a single session or has it spread? Has the model's behaviour been persistently altered? Are other agents using the same MCP servers affected? The triage phase should produce a preliminary severity classification that determines the urgency and scope of the response: critical for confirmed data exfiltration or persistent model compromise, high for successful prompt injection with tool misuse, moderate for anomalous behaviour without confirmed impact.

Phase 2-3: Containment and Forensic Analysis

Containment for AI breaches must be fast and decisive while preserving forensic evidence. Immediate containment actions include revoking all authentication tokens for the affected agent sessions, isolating the MCP server from the network using pre-configured kill-switch mechanisms, suspending the affected agent role across all sessions to prevent the attack from spreading, and preserving all conversation logs, tool invocation records, and model outputs before they're rotated or overwritten. If the incident involves a compromised third-party MCP server, disconnect it from your infrastructure immediately and notify other organisations that may be using the same server. Forensic analysis for AI breaches requires specialised techniques. Reconstruct the full conversation timeline from the earliest anomalous event, mapping every tool invocation, its parameters, and its results. Identify the injection vector by analysing the conversation content that preceded the anomalous behaviour—was the injection in a user message, a tool output, a retrieved document, or cached conversation history? Determine the scope of data exposure by cross-referencing tool invocation logs with backend system access logs, identifying every piece of data the agent accessed during the compromised session. Evaluate whether the model's behaviour has been persistently altered by running a standardised evaluation suite that tests for known prompt injection artifacts. Document every finding with timestamps, evidence references, and chain of custody information that will stand up to legal and regulatory scrutiny.

Phase 4-5: Recovery and Eradication

Recovery from an AI breach requires addressing both the immediate vulnerability that was exploited and the systemic weaknesses that allowed the breach to occur. Begin eradication by deploying specific mitigations for the identified attack vector: if the breach resulted from prompt injection, update your input sanitisation rules to detect the specific payload pattern, add the attack signature to your monitoring system's detection rules, and implement output validation controls that would have caught the anomalous tool invocations. If the breach involved a compromised MCP server, remove it permanently and replace it with a vetted alternative or disable the affected capability. Rotate all credentials associated with the affected infrastructure, including not just the directly compromised tokens but any credentials that the compromised agent could have accessed through its tools. Recovery should proceed in controlled stages: first, restore services with enhanced monitoring and restricted permissions that limit agents to a minimal tool set; second, gradually re-enable tools as confidence builds that the vulnerability has been fully addressed; third, return to full operations while maintaining elevated monitoring for a cooldown period of at least thirty days. Throughout recovery, maintain detailed records of every action taken, every configuration changed, and every credential rotated, creating a comprehensive remediation log that demonstrates due diligence to regulators and auditors.

Phase 6: Post-Incident Review and Improvement

The post-incident review is where the lasting value of every incident is captured. Conduct a blameless retrospective within seventy-two hours of the incident's resolution, bringing together the IR team, the engineering team responsible for the affected MCP infrastructure, the security monitoring team, and relevant stakeholders from compliance and management. The review should produce a detailed timeline of the incident from initial compromise to full recovery, an analysis of what detection mechanisms worked and what gaps allowed the breach to proceed undetected, an evaluation of containment effectiveness including the time from detection to containment and whether the containment actions were appropriate and sufficient, an assessment of recovery speed and completeness, and a prioritised list of improvements to prevent similar incidents. The improvement list should include specific technical changes such as new detection rules, updated sanitisation patterns, or additional access controls, process changes such as updated escalation procedures or communication templates, and training needs identified during the incident. Assign each improvement an owner and a deadline, and track implementation through your standard project management process. Update the incident response playbook with new procedures, decision criteria, and lessons learned. The most valuable outcome of any incident is a measurably stronger security posture that makes the next incident less likely and the next response more effective.