AI incidents — what counts and what to log
What actually counts as an AI incident in 2026 — with worked examples, the EU AI Act Art. 73 serious-incident test, and the log structure reviewers expect.
What counts — worked examples
- Hallucination with consequence — a customer-support LLM tells a buyer your warranty covers something it does not, and you honour it. Yes, log it.
- Prompt injection — a CV uploaded to your ATS contains hidden instructions that bias the screening LLM. Yes, OWASP LLM01:2025.
- Sensitive data in prompts — an employee pastes a customer list into a free ChatGPT account. Yes — GDPR personal-data incident regardless of harm.
- Bias drift — quarterly fairness test shows the model now under-approves a protected group by 11pp vs baseline. Yes, log and remediate.
- Agent runaway — an autonomous agent makes 400 API calls in a loop and bills $9k. Yes, governance and cost incident.
- Vendor breach — your LLM vendor discloses a training-data exposure that includes your prompts. Yes, and notify customers per your DPA.
The minimum log structure
- Incident ID, date detected, date occurred (often different).
- System ID, model version, prompt template version.
- Severity (Low / Med / High / Serious per EU AI Act Art. 3(49)).
- Affected parties and data categories.
- Root-cause classification (model, data, prompt, user, vendor, agent).
- Containment actions, remediation, owner, due date.
- Regulatory clock: GDPR 72h, EU AI Act 15 days for serious, narrower for fundamental-rights breach.
Do this Monday
- Create an "AI incidents" Jira / Linear project with the schema above as the issue template.
- Wire your AI gateway (LiteLLM, Portkey, Azure AI Content Safety) to auto-open a ticket on safety filter triggers.
- Run a tabletop on the prompt-injection example with engineering + legal.
Reviewer hot-buttons
- A log that shows zero incidents is itself a finding — it means you are not detecting them.
- Severity must be defined; "serious" must reference EU AI Act Art. 3(49) wording.
- Closure must show what changed, not just "fixed".