Nis2 on lo0 — Tech Blog

Hardening and secrets in the sovereign LLM stack: defence in depth

Thu, 11 Jun 2026 11:00:00 +0000

Part of the operational series on squeezing a generic on-premise 4×H100 SXM 80GB LLM cluster. The sibling pieces: the document ingestion from PDF to indexed chunk that fills the vector database, the embeddings and rerankers service with TEI in production that feeds it, and, the most directly coupled to this post, the GitOps of the inference stack with Flux, because GitOps and secrets share a chicken-and-egg problem we solve here. Assembling all of this into a conversational assistant (LibreChat + LiteLLM + RAG) is covered by another post in the series, still in draft.

TL;DR

A sovereign LLM inference stack is, at minimum, six services: a gateway (LiteLLM or equivalent), an inference engine (vLLM), an embeddings/rerankers service (TEI), a vector database, a state database (conversations, users) and a front end (LibreChat). Six services are six attack surfaces, and the assistant “works” as soon as the gateway returns tokens, long before it is hardened. This post walks through defence in depth layer by layer:

Secrets: never in the clear in git. sealed-secrets (asymmetric encryption, controller in the cluster) vs SOPS/age (file encryption, simple, pure GitOps) vs External Secrets Operator + Vault (dynamic secrets, native rotation). GitOps’s chicken-and-egg problem.
Network: default-deny NetworkPolicy + explicit allow; L3/L4 with the standard NetworkPolicy, L7 and DNS-based egress with Cilium (eBPF); egress control so the data does not leave the perimeter; internal mTLS.
Pod security: runAsNonRoot, readOnlyRootFilesystem, dropped capabilities, seccomp RuntimeDefault, restricted Pod Security Standards, nothing privileged.
Supply chain: pin by digest, scan with Trivy, sign with cosign, admission control that rejects anything unsigned.
AuthN/Z: LiteLLM virtual keys, OIDC/LDAP in the front end, Kubernetes RBAC with least privilege.
Runtime: detection and, optionally, enforcement with Tetragon.
Data at rest: storage encryption, credentials for the vector database and the state database.

The thesis: hardening reduces the blast radius, it does not eliminate it. That is why you prioritise by impact, and the first job on a 4×H100 is the gateway↔vector store pair and egress-deny.

The analogy: the sovereign office

Picture an office holding sensitive documentation: the corpus that feeds the RAG, the users’ conversations, the engine’s credentials. It is not enough for the street door to lock. Real security is defence in depth: several layers, each assuming the previous one can fail.

The mapping is direct:

The building’s perimeter is egress control: making sure a leaked secret or a compromised process cannot exfiltrate the corpus to an external server. It is the most underrated layer and the first one I put in place in a sovereign deployment.
Access control at every interior door is the default-deny NetworkPolicy: the front end does not talk to the vector database directly because it has no reason to; only the strictly necessary pairs are open.
The fire partitions are pod isolation (restricted) and namespaces: if one service catches fire, the fire does not jump to the one next door.
The doorman who checks credentials is the admission control that verifies each image’s cosign signature before letting it into the cluster.
The cameras are Tetragon/Falco: they record what each process did and detect (or kill) anything anomalous.
The safe with a rotated key is the secrets: encrypted at rest, out of git, and rotated so a stolen key expires.

No layer is sufficient on its own. The building is secure because crossing them all is expensive.

Layer 1 — Secrets: GitOps’s chicken-and-egg problem

The stack is deployed by GitOps: the sibling piece on Flux reconciles the state declared in git against the cluster. That is excellent for manifests, but there is a foundational problem: the inference engine needs the Hugging Face token to download the model, the state database needs its password, the gateway needs its master key. Those secrets cannot go into git in the clear. Anyone with read access to the repo, and in an organisation with generous read that is a lot of people, plus every repo backup, plus every fork, would read them.

The chicken and the egg: GitOps wants all state to be in git, but secrets cannot be in git in the clear. The solution comes in three families:

Family A — sealed-secrets (asymmetric encryption, controller in the cluster)

Bitnami Sealed Secrets is a controller running in the cluster plus a client CLI, kubeseal. It uses asymmetric cryptography: there is a key pair. The public one is held by developers and is used to encrypt; the private one lives only in the cluster’s controller and is used to decrypt. The flow:

The developer takes a normal Secret and encrypts it with kubeseal, which obtains the controller’s public key. Result: a SealedSecret resource.
That SealedSecret is committed to git in the clear — it is encrypted, it is not readable.
The controller in the cluster detects it, decrypts it with its private key and creates the real Kubernetes Secret in the target namespace.

The key property: because the private key never leaves the cluster, neither the developer nor anyone with access to git can decrypt. And the encryption includes the namespace name: a SealedSecret sealed for inferencia cannot be moved to front and decrypted there — it behaves as if each namespace had its own key. The controller also manages rotation of the sealing keys, labelling them as active or compromised.

Advantage: it fits pure GitOps perfectly, the encrypted secret lives with the rest of the state. Limitation: the secret, once decrypted, ends up as a normal Kubernetes Secret — it sits in etcd, and etcd has to be encrypted separately (we will see that in layer 7).

Family B — SOPS + age (file encryption, simple)

SOPS encrypts the whole YAML/JSON file (or only its values) and leaves it in git encrypted, decrypting it only at deployment time. It supports cloud KMS (AWS/GCP/Azure), PGP and, the relevant one for on-premise sovereignty, age, a simple serverless offline encryption scheme. The GitOps operator (Flux brings native SOPS integration) decrypts on reconciliation using the age key stored in the cluster.

Advantage: simple, scriptable, with no server to maintain; the operational weight falls on safeguarding the age key. It is the usual recommendation for small teams or for getting started. Limitation: rotation is manual (re-encrypt everything with the new key) and there are no dynamic secrets.

Family C — External Secrets Operator + Vault (dynamic secrets, native rotation)

The External Secrets Operator (ESO) keeps no secrets in git at all. All that goes into git is an ExternalSecret: a reference saying “the password field comes from path secret/data/vectordb in such-and-such a SecretStore”. The operator reads from an external store, typically HashiCorp Vault, and synchronises the Kubernetes Secret. Vault keeps the secrets in its own encrypted backend and serves them over an authenticated API, so that they never reside in git and, with its dynamic engines, it can issue short-lived database credentials that expire on their own.

Advantage: native rotation, dynamic secrets, centralised auditing, a single point of governance. Limitation: you have to operate Vault (sealing/unsealing, high availability, access policy), which is real work.

How to choose, and rotation

Criterion	sealed-secrets	SOPS + age	ESO + Vault
Secret in git	encrypted	encrypted	reference only
Extra server	lightweight controller	none	Vault (heavy)
Rotation	automatic sealing keys	manual (re-encrypt)	native / dynamic
Dynamic (short-lived) secrets	no	no	yes
Fit with pure GitOps	excellent	excellent	good (references)
Operational curve	low	very low	high

For a sovereign 4×H100 getting started, SOPS/age or sealed-secrets cover 90% at minimal cost. When the system grows and requirements for frequent rotation and auditing appear, typical in ENS High Category scenarios, you migrate to ESO + Vault. The rotation rule: a secret that never rotates is a secret that, once leaked, is leaked forever. Rotation does not prevent the leak; it bounds its window of validity. We will come back to that when we talk about blast radius.

Layer 2 — Network: default-deny, egress and the arithmetic of the surface

The arithmetic of communication pairs

Without a network policy, in Kubernetes every pod can talk to every pod. That is the default posture, and it is the worst one for a sovereign system. With $N$ services, the number of possible ordered communication pairs (who-can-call-whom) is:

$$P_{\text{abierto}} = N \cdot (N-1) \approx N^2$$

With our $N = 8$ components (gateway, vLLM, TEI, vector database, state database, front end, plus the secrets controller and the observability one), that is $8 \cdot 7 = 56$ possible directed pairs. Fifty-six paths along which a compromised service could pivot laterally.

Now we apply a default-deny NetworkPolicy: nothing talks to anything except what is explicitly allowed. The real whitelist of an LLM stack is small. The strictly necessary pairs:

front end → gateway
gateway → vLLM
gateway → TEI
gateway → vector database
gateway → state database
front end → state database (sessions/users)

That is $E = 6$ edges. The communication surface falls from $56$ to $6$:

$$\frac{E}{P_{\text{abierto}}} = \frac{6}{56} \approx 0.107$$

Almost 89% of the possible paths are closed. The front end can no longer touch vLLM or TEI or the vector database directly; if somebody compromises the front end, they have no network route to the corpus. This is the difference between $\sim N^2$ and a whitelist $E \ll N^2$.

Standard L3/L4, L7 and egress with Cilium

Kubernetes’ native NetworkPolicy operates at L3/L4: it selects pods by label and allows/denies by port and protocol. That covers the bulk of the whitelist. But it has limits: it does not understand DNS or HTTP. This is where Cilium comes in, applying policies in the kernel via eBPF and extending them with CiliumNetworkPolicy:

Egress by FQDN/DNS: instead of pinning IPs (which change), allow egress only to huggingface.co so vLLM can download the model and block the rest. Critical for egress control.
L7: allow only certain HTTP methods/paths between the gateway and an internal service.
Enforcement modes: in the default mode, an endpoint with no policy selecting it has everything open; in always mode, everything is denied until a policy explicitly opens it. For a sovereign system, always + default-deny is the target posture.

Egress control is the sovereign piece par excellence. An internal service being unable to open connections to the internet except to a handful of explicit destinations means that, even if vLLM or the gateway is compromised, the corpus cannot leave the perimeter. The RAG is the asset of value; egress-deny is what stops it leaking out.

Internal mTLS

The NetworkPolicy says who can talk to whom, but it neither encrypts nor authenticates east-west traffic. For that, mTLS (mutual TLS: client and server authenticate each other). Cilium offers native mutual authentication; alternatives such as Linkerd or Istio give you a full service mesh. For a six-service stack, Istio’s full mesh is usually over-engineering; Cilium’s mutual authentication or Linkerd (lighter) is proportionate. The effect: an attacker on the cluster network cannot impersonate the gateway or eavesdrop on internal traffic in the clear.

Layer 3 — Pod security: restricted, and nothing privileged

The Pod Security Standards define three profiles: privileged (no restrictions), baseline (the reasonable minimum) and restricted (hardened). The target for the whole LLM stack is restricted, applied via Pod Security Admission with a namespace label in enforce mode. The restricted profile requires:

runAsNonRoot: true and a runAsUser other than 0. No container runs as root.
readOnlyRootFilesystem: true. The root filesystem is read-only; anything that needs to write goes to an emptyDir or an explicit volume. An attacker cannot leave persistent binaries in the container.
drop: ["ALL"] of Linux capabilities. No capabilities that are not needed.
seccomp RuntimeDefault, which applies the runtime’s profile and blocks dangerous syscalls.
allowPrivilegeEscalation: false, nothing privileged, no host namespaces.

The nuance with GPUs: pods using an H100 load NVIDIA’s device plugin, which historically tempted people to relax the securityContext. You do not need to run the inference container as privileged in order to use the GPU; device access is managed by the plugin, and the vLLM pod can and should run restricted. This is a fire partition: if vLLM is compromised, the attacker has a non-root process, with no capabilities, a read-only FS and trimmed syscalls — a terrible starting point for escalation.

Layer 4 — Image supply chain

The container image is third-party code running with access to your data. Three composed controls:

Pin by digest, not by tag. vllm/vllm-openai:latest is a moving target; @sha256:... is immutable. Pinning the digest guarantees that what was deployed is exactly what was audited.
Scanning with Trivy. Before promoting an image, Trivy enumerates its CVEs and composes the SBOM (component inventory). The pipeline fails if there are unmitigated critical vulnerabilities.
Signing with cosign and verification at admission. The Sigstore project (cosign for signing, Fulcio as a CA for ephemeral certificates via OIDC, Rekor as an immutable transparency log) allows the image to be signed. And Sigstore’s policy-controller, or Kyverno, is an admission controller that verifies the signature before admitting the pod: an image without a valid signature does not enter the cluster.

This is the doorman from the analogy: he checks each image’s credential (signature) at the door. A poisoned image uploaded to a registry, or a hijacked tag, stays outside because it carries no signature from the trusted issuer.

Layer 5 — AuthN/Z: virtual keys, OIDC and minimal RBAC

Three identity control points, from the outside in:

Front end (OIDC/LDAP). The human user authenticates against the corporate identity provider. The front end does not invent its own user system; it delegates to OIDC. This gives SSO, MFA and centralised revocation.
Gateway (LiteLLM virtual keys). The gateway issues virtual keys: each team, application or user has its own key with a budget, a rate limit and permitted models. The gateway’s master key, and the real API keys towards the engine, are never seen by the client; clients only handle their virtual key, revocable individually. If a virtual key leaks, that one and only that one is revoked.
Kubernetes RBAC with least privilege. Each service’s ServiceAccount has exactly the permissions it needs. The vLLM pod does not need to list Secret objects in other namespaces or create pods. Restrictive RBAC means a stolen service account token opens very little.

Layer 6 — Runtime: the cameras with Tetragon

The previous layers are preventive. What is missing is detection: what if something, despite everything, runs where it should not? Tetragon is observable security and runtime enforcement over eBPF, Kubernetes-aware. It hooks kernel events — process_exec, tcp_connect, security_file_open — with typical overhead below 1%, and it can move from observing (recording the event) to enforcement (killing the process or cutting the connection in the kernel, Sigkill).

The operational rule, which I set out in detail in the bubblewrap + Tetragon runbook: adopt first, block later. First you deploy in observation mode to build a baseline of the stack’s normal behaviour — which binaries vLLM executes, what the gateway connects to — without false positives. Only afterwards do you promote the clear rules to enforcement: kill any process that tries to read secret paths, or cut every tcp_connect to destinations outside the egress whitelist. Falco is the detection-only alternative over eBPF; Tetragon adds enforcement. These cameras also produce the audit evidence: what each service ran and which attempt was blocked.

Layer 7 — Data at rest

The last thing we protect is data standing still:

Storage encryption. The persistent volumes, where the vector database and the state database live, on top of disk encryption (LUKS/dm-crypt) or Ceph-level encryption. A disk stolen from the datacenter does not reveal the corpus.
etcd encryption. Remember that Kubernetes Secret objects, once decrypted by sealed-secrets or ESO, are normal objects in etcd. You have to enable etcd encryption at rest, or the secret sits in the clear in the control plane.
Database credentials. The vector database’s password and the state database’s are first-class secrets (layer 1), never embedded in the manifest or in environment variables in the clear in git.

Exposure table: service × port × external egress?

This table is the input for writing the NetworkPolicy resources. The egress column is the one that decides what leaves the perimeter.

Service	Internal port	Permitted callers	External egress?
front end	3080	(human ingress via OIDC)	No
gateway (LiteLLM)	4000	front end	No
vLLM	8000	gateway	Only `huggingface.co` for the initial download; zero in operation
TEI (embeddings/reranker)	8080	gateway	Only model download; zero in operation
vector database	6333	gateway	No
state database	5432	gateway, front end	No
secrets controller	—	(control plane)	No
observability	9090	internal scraping	No

The reading: no service needs external egress in normal operation. vLLM and TEI need it once to pull the model, and that permission can be temporary or restricted by FQDN to huggingface.co. Everything else is total egress-deny. If a service starts attempting outbound connections this table does not contemplate, Tetragon records it and, in enforcement, cuts it.

Blast radius: a leaked secret, with and without defence

The blast radius measures how much damage a compromise does. Let us model it for the realistic worst case: the state database’s credential leaks.

Without hardening (flat network, unrotated secret, no egress-deny, no detection):

The secret gives access to the state database from any pod (flat network → 56 open pairs).
The secret does not rotate → it is valid indefinitely; the exploitation window is $\infty$ until somebody notices.
Without egress-deny, the attacker dumps the whole database to an external server.
Without Tetragon, nobody finds out until the public incident.
Blast radius: the entire state database, exfiltrated, undetected, for an indefinite time.

With hardening (default-deny, ESO + Vault with dynamic credentials, egress-deny, Tetragon in enforcement):

The secret is only usable from the pod that has a network route to the state database (1-2 pairs, not 56).
With Vault’s dynamic credentials, the secret expires — say within a window $T$ of hours; past $T$, it is worthless.
Egress-deny prevents the dump to the internet: the attacker can read, but cannot take anything out.
Tetragon records the anomalous access and, in enforcement, kills the process attempting the outbound connection.

The qualitative reduction is enormous, but let us be quantitative about the window. If a static secret is valid forever and a rotated one with period $T$ is valid for at most $T$, and compromises arrive at rate $\lambda$, the expected number of live and exploitable secrets at any given instant goes from growing without bound to being capped at $\lambda \cdot T$. With rotation every 24 h ($T = 1$ day) against “never”, a given secret’s window falls from months to a day: a reduction of one to two orders of magnitude in temporal exposure. Combined with the reduction in network pairs ($56 \to \sim 2$, a factor of $\sim 28\times$) and egress-deny (from exfiltration possible to impossible by the direct route), the blast radius is drastically reduced.

But let us be honest: it does not reach zero. The attacker with the credential can read the state database during the window $T$ from the compromised pod. Hardening turned “indefinite, silent catastrophe” into “a bounded, detected incident with no exfiltration by the direct route”. That is exactly what defence in depth promises: not invulnerability, but that the cost of a full compromise is high and its radius small.

The ENS / NIS2 angle

These layers are not voluntary hygiene: they implement specific controls. The mapping is developed in detail in the technical controls ENS × ISO 42001 × EU AI Act post; here, the actionable summary:

Hardening layer	ENS measure (RD 311/2022)	NIS2 / framework link
Encrypted secrets + rotation	`op.exp.11` (cryptographic keys), `mp.info.3` (encryption)	Credential management; in High Cat., HSM
Default-deny NetworkPolicy + segmentation	`mp.com.1` (perimeter), `mp.com.4` (flow separation)	NIS2 art. 21: network security measures
Egress control	`mp.com.1` + `op.mon.1`	Exfiltration prevention
Internal mTLS	`mp.com.2-3` (confidentiality/integrity in transit)	Mandatory TLS
Restricted Pod Security	`op.exp.2` (hardened configuration)	Configuration hardening (CIS)
Trivy + cosign + admission	`op.exp.6` (malicious code), `op.ext.3` (supply chain)	NIS2 supply chain
OIDC + virtual keys + RBAC	`op.acc.1-2-5` (identification, access, authentication)	MFA in High Cat.
Tetragon runtime	`op.mon.1` (intrusion detection)	Monitoring and response
Encryption at rest + etcd	`mp.info.3` (encryption), `mp.si` (media)	Data at rest

The honest note for auditing: hardening reduces risk, it does not eliminate it, and maturity is demonstrated by prioritising by impact. A competent auditor does not want to see nine half-finished layers; they want to see that egress-deny on the critical asset (the corpus) and secrets management are solid before mTLS is perfect between low-sensitivity services. For the management and governance context, see also ISO/IEC 42001 as an AIMS and the EU AI Act mapping onto the architecture.

Applied to the generic 4×H100 cluster

In a real deployment on 4×H100 SXM 80GB, you do not harden the nine layers at once. The order, prioritised by impact:

Egress-deny on vLLM, TEI and the vector database, first. It is the barrier that stops the corpus leaving the perimeter, and it goes in as soon as the services start. Allow egress by FQDN to huggingface.co only during the model’s initial download; afterwards, zero. It is the layer with the highest return per hour invested.
Secrets for the inference engine and the databases, out of git. The Hugging Face token vLLM uses to download the model, the vector database’s and the state database’s passwords, the gateway’s master key: into sealed-secrets or, if there is already a rotation requirement, into ESO + Vault. Never in values.yaml in the clear.
Default-deny NetworkPolicy + the 6-edge whitelist. The gateway and the vector database are the most exposed services — the gateway because it receives all the traffic, the vector database because it holds the embedded RAG. Closing everything that is not on the whitelist cuts lateral pivoting from $\sim N^2$ to the 6 real edges.
restricted Pod Security in the inference namespace, including the vLLM pod with a GPU (it needs no privileges to use the H100).
cosign + admission so only signed images get in; Trivy in the GitOps pipeline of Flux.
Tetragon in observation, baseline, and then enforcement over the egress of the sensitive pods.
Encryption at rest for the databases’ volumes and for etcd.

The gateway and the vector database are the two I harden first inside the whitelist: the gateway for being the exposed face, the vector database for holding the asset that egress-deny protects. The rest is built on top, layer by layer, always assuming the previous one can fail.

What hardening does NOT solve

To close honestly, what these layers do not cover and what needs other pieces:

Prompt-level attacks (jailbreak, indirect injection via the RAG corpus): that is the job of guardrails and LLM Guard, not of NetworkPolicy.
Agents with legitimate permissions doing something harmful: the runtime isolation from the agent isolation post bounds what an agent can touch, but a granted permission is a usable permission.
The human factor: a well-managed secret shared over Slack is still leaked. Rotation bounds the window, it does not eliminate the mistake.
Zero-day vulnerabilities in the components themselves: Trivy detects what is known; the unknown gets through until the CVE is published.

Hardening is a multiplier on the cost of attacking, not an impassable wall. Its value lies in making an individual compromise bounded, detected and with no way out — and that, for a sovereign system holding sensitive data, is exactly the goal.

Incident response runbooks for LLM inference: every alert to a concrete action with Kafka and Keep

Tue, 02 Jun 2026 04:30:00 +0200

This post closes the observability trilogy opened by GPU observability for LLM inference (which metrics) and Anatomy of the twelve DCGM and five vLLM metrics (which documented anomaly per metric). Here each anomaly gets its concrete action and is fitted into the incident management machinery that compliance demands.

TL;DR

The GPU observability alerts are useless without a codified procedure for each one; the operator who interprets them by hand every time is operating on intuition. The right combination has three indispensable pieces. (1) Runbook catalogue: for each of the six critical alerts (GpuHbmNearOom, GpuThermalOrPowerThrottle, GpuXidErrorDetected, GpuEccDoubleBit, VllmKvCachePoolNearFull, VllmTtftP95OutOfSlo), a severity, an immediate mitigation, the evidence to capture before remediating, a resolution action, a closure criterion and a postmortem trigger. (2) Reproducible pipeline: Prometheus + DCGM → Alertmanager → Kafka as event bus (topics gpu.alerts.enriched, incidents.lifecycle, audit.actions with WORM retention) → Keep as workflow engine (declarative YAML workflows versioned in git) → Kubernetes job / script / ChatOps executors. (3) Formal fit into incident management under the regulatory corpus: ISO/IEC 27035 phases identify → report → assess → respond → learn; ENS controls op.exp.7 (incident management), op.exp.8 (activity logging), op.exp.10 (notification to users); NIS2 art. 23 with an early warning at 24 h, a formal notification at 72 h and a final report at 1 month; EU AI Act art. 73 for a serious incident on a high-risk system, deadlines of 2 to 15 days depending on severity; ISO/IEC 42001 clause 10 (continual improvement of the AIMS). The action taxonomy is immediate mitigation (drain, throttle, scale-down: contains the damage in seconds) → diagnosis (evidence capture with nvidia-smi -q, dmesg, a vLLM /metrics snapshot, the related OTel trace; without this the postmortem is not defensible) → resolution (restart, reset, RMA, rollback) → postmortem (5-whys RCA, prevention plan, runbook update). Kafka contributes the immutable audit trail that ENS and the EU AI Act demand: every action executed by Keep or by a human is published as an event on audit.actions with timestamp, actor, decision and evidence, retained WORM for at least 6 months. Keep contributes workflows as code: this post includes three complete workflows (XID with drain plus Jira ticket, ECC DBE with immediate paging and node blocking in the scheduler, automatic canary rollback on TTFT P95 out of SLO). Four anti-patterns close the material: alerts without a runbook (most of them), a runbook without prior evidence capture (it perpetuates the incident because the root cause is lost), escalation by seniority rota instead of severity (a junior operator handling an ECC DBE), and the absence of a human gate for destructive actions (Keep running nvidia-smi --gpu-reset without confirmation). Applicable to a generic 4×H100 SXM cluster with Kafka and Keep already deployed.

You are here: OBSERVE → DEPLOY (incident response closes the loop)

The analogy: a nuclear reactor control room

In a nuclear plant control room, the operator on shift never decides what to do on seeing an alarm. The decision is pre-made and codified in a written procedure (SOP) covering every alarm on the panel: if X sounds, open book X, read steps 1-N, execute exactly, call the supervisor at step M, escalate to the plant director at step N+3. The reason is strict: critical alarms are rare but catastrophic if handled badly; an operator improvising in an emergency makes worse decisions than one applying a procedure reviewed by experts and validated by simulation.

The reactor does not expect the operator to be a genius. It expects them to know the procedures to the letter and the operations management system to hand them the right procedure at the right moment. If the procedures are not written, not versioned, or not integrated with the alarms that fire them, the control room is operating on intuition. The difference between the two operations, procedural versus intuitive, is the difference between a plant that runs 30 years without incidents and one that ends up on a blacklist.

Incident response on an LLM inference cluster works identically. The DCGM and vLLM alerts listed in the previous posts are the panel alarms. Each one needs its written SOP, versioned, integrated with the alert that fires it and reviewed after every incident. Without that codification, the operator on shift improvises in the middle of an ECC DBE failure at 4 in the morning; with it, they execute the nine steps of runbook 12 and the incident closes in 20 minutes.

The incident pipeline architecture

Prometheus + DCGM. Collects the metrics described in the two previous posts. PrometheusRules define the six critical alerts with for: <duration> to avoid noise.

Alertmanager. Receives raw alerts; deduplicates, groups by labels ({cluster, node, gpu, model}), routes. Instead of sending straight to PagerDuty or Slack, it sends to Kafka via a webhook receiver. That turns the alert into a bus event that multiple consumers process (Keep for action, the audit topic for compliance, dashboards for visualisation).

Kafka as event bus. Three canonical topics:

gpu.alerts.enriched — alerts with added context (tenant, model, version, namespace owner, effective severity). Retention: 7 days, replication factor 3.
incidents.lifecycle — incident cycle events: incident.opened, incident.acknowledged, action.proposed, action.executed, incident.escalated, incident.resolved, postmortem.attached. Retention: 90 days.
audit.actions — an immutable record of every action executed (by Keep automatically or by a human confirming). Retention: 6 months minimum with compaction off plus tiered storage, WORM storage. This is the topic ENS op.exp.8, EU AI Act art. 12 and NIS2 require you to keep.

Keep as workflow engine. Consumes from gpu.alerts.enriched, fires YAML workflows versioned in git, executes actions (HTTP calls, kubectl jobs, Slack messages, Jira tickets) and publishes the result to incidents.lifecycle plus audit.actions. Choosing Keep over Alertmanager alone (or over PagerDuty alone) is deliberate: Keep separates runbook declaration (readable, reviewable YAML) from notification delivery (PagerDuty). The runbook is versioned code; notifications are operational details.

Executors. What actually moves the cluster:

Kubernetes jobs: kubectl drain, kubectl cordon, kubectl rollout undo.
NVIDIA API: nvidia-smi --gpu-reset, dcgmi diag -r <level>.
ChatOps: human confirmations through Slack interactive messages before a destructive action runs.
External tooling: Jira ticket, PagerDuty notification, CMDB call.

The six critical alerts and their runbooks

For each alert: severity, immediate mitigation (seconds), evidence to capture before remediating, resolution action, closure criteria, postmortem trigger.

RB-01 · `GpuHbmNearOom` — HBM > 92 % sustained

Severity: WARNING. Risk of OOM on the next PagedAttention allocation.

Immediate mitigation. Reduce admission temporarily by lowering max_num_seqs on the affected engine via hot reload (if the engine supports it) or a staggered restart of replicas. Trigger an additional scale-out via KEDA if there are free GPU nodes. There is no need to drain the node.

Evidence to capture.

nvidia-smi --query-gpu=index,memory.used,memory.free,memory.total --format=csv
nvidia-smi -q -d ROW_REMAPPER | grep -i pending
curl http://vllm-pod:8000/metrics | grep -E "gpu_cache_usage|num_requests"
kubectl logs <pod> --tail=200 | grep -i "preempt\|swap"

Save the snapshot to audit.actions with a timestamp and an incident_id.

Resolution. If the cause is a traffic spike: let the autoscaler scale to a stable regime, monitor for 30 min. If the cause is a model regression (canary v2 consumes more KV cache than v1): roll the canary back (see RB-06). If it is a leak (the metric grows without traffic growing): restart the pod with a heap dump capture.

Closure. gpu_cache_usage_perc < 80 % sustained for 15 min AND num_requests_waiting == 0.

Postmortem. Not mandatory unless the incident lasted > 30 min or had SLO impact.

RB-02 · `GpuThermalOrPowerThrottle` — bit ≠ 0 and not Idle in CLOCK_THROTTLE_REASONS

Severity: WARNING (thermal) or CRITICAL (sustained HW Power Brake, PDU risk).

Immediate mitigation. Identify the bit (decode the bitmap). If it is 0x40 HW_THERMAL or 0x20 SW_THERMAL: drain the workload from the node to other replicas if the temperature does not drop within 2 min, and stop new pods landing on that node (kubectl cordon). If it is 0x80 HW_POWER_BRAKE: alert DC infrastructure immediately (likely an over-committed PDU, the Dell KB 000220508 / Lenovo HT514380 case) and lower the TDP of the rack’s GPUs via nvidia-smi -pl to a lower value to relieve load on the breaker.

Evidence.

nvidia-smi --query-gpu=index,temperature.gpu,temperature.memory,power.draw,clocks_throttle_reasons.active --format=csv
ipmitool sdr | grep -i "fan\|temp\|inlet"
# PDU data if they are instrumented (modbus / SNMP)

Resolution.

Thermal: review rack airflow, check the rear-door HX, T_inlet, DGX fans. An infra issue, not an engine one.
Power Brake: review the sizing of the PDU branch, the breaker, the 415 VAC distribution. Likely a redistribution of load to another branch or a temporary TDP cap.

Closure. CLOCK_THROTTLE_REASONS == 0x1 (Idle only) or 0x0 for 30 min under normal load.

Postmortem. Mandatory if it was a HW Power Brake: that implicates the DC electrical infrastructure.

RB-03 · `GpuXidErrorDetected` — `increase(DCGM_FI_DEV_XID_ERRORS[5m]) > 0`

Severity: CRITICAL.

Immediate mitigation. kubectl cordon the node (no more new pods). If the XID is 31/48/79/94/95 (hardware or cascade): drain the node’s existing pods. If the XID is 13/43 (possibly software): keep the pods but block new ones, capture the trace and the active workload.

Evidence.

# The specific XID from dmesg
dmesg | grep -i xid | tail -30
nvidia-smi -q -d ERROR
nvidia-smi -q -d PCIE
# State of the retired pages
nvidia-smi -q -d ROW_REMAPPER
# Workload that was running
kubectl get pods -o wide | grep <node>
kubectl logs <pod> --previous --tail=500

Resolution.

XID 13/43 (software exception / channel verif): if it recurs only with one specific model, it is a workload bug; raise an issue with the model team. If it is transient, restarting the pod is enough.
XID 31 (MMU fault): usually a cascade from a previous XID 48. Reset the GPU (nvidia-smi --gpu-reset -i <index>) or reboot the node if the reset does not resolve it.
XID 48 / 95 (DBE / uncontained ECC): see RB-04. The node goes into quarantine.
XID 79 (fallen off the bus): reboot the node. If it recurs after the reboot, open an RMA for the GPU. ByteDance reports 43 % co-occurrence with PCIe errors, so check the slot and the cable too.
XID 94 / 145 / 149: catalogued in NVIDIA’s Xid Catalog with a specific procedure.

Closure. Node smoke test passed (dcgmi diag -r 3), 24 h with no new XIDs, back into the pool.

Postmortem. Mandatory. Include the specific XID, the distribution of XIDs across the cluster, an updated MTBE.

RB-04 · `GpuEccDoubleBit` — `DCGM_FI_DEV_ECC_DBE_VOL_TOTAL > 0`

Severity: CRITICAL, data corruption in progress.

Immediate mitigation. Drain the node immediately without waiting for further evidence. Page (PagerDuty / OpsGenie) the primary ON-CALL. Mark the node unschedulable and failed. XID 48 has a 100 % probability of killing the running job according to the Story of Two GPUs dataset; any inference already in flight is compromised.

Evidence (in parallel with the mitigation).

nvidia-smi -q -d ECC
nvidia-smi -q -d ROW_REMAPPER # Pending: Yes expected
dmesg | grep -E "Xid.*48|DBE|double-bit" | tail -50
# Full capture of the GPU state
dcgmi diag -r 4 -i <gpu_index>

Resolution. Full GPU reset (nvidia-smi --gpu-reset) or a node reboot if the reset does not complete. The reset activates the row remap. After the reboot:

nvidia-smi -q -d ROW_REMAPPER # Pending: No expected
nvidia-smi -q -d ECC # volatile counters at 0

If RETIRED_DBE > 8 pages after the remap: schedule a GPU replacement in the next window, because silicon degradation is progressive. The published real case documents ~19 hours of downtime as typical.

Closure. Node back in the pool after 48 h with no new DBEs.

Postmortem. Mandatory. If the incident affected a request carrying personal or classified data, evaluate notification to the DPO under GDPR art. 33 (it is not necessarily a breach, but it has to be assessed).

RB-05 · `VllmKvCachePoolNearFull` — `gpu_cache_usage_perc > 95 %` sustained 3 min

Severity: WARNING (risk of preempt-on-OOM, not of a real OOM).

Immediate mitigation. Activate autoscaler scale-out by lowering the KEDA threshold temporarily (from 0.85 to 0.75) for 30 min. In recompute mode, preempts raise TTFT but do not break requests; acceptable in the short term. In swap mode latency goes through the roof, so it is better to cut new traffic (return 503 from the router) for 5 min.

Evidence.

curl http://vllm-pod:8000/metrics | grep -E "gpu_cache|num_requests|num_preemptions"
kubectl get hpa vllm-llama70b
kubectl logs <pod> --tail=200 | grep -i preempt

Resolution. If it recurs regularly: revisit capacity planning, possibly lowering max_num_seqs or raising the stable replica count. See Capacity planning.

Closure. Pool < 85 % sustained 30 min, no preempts in the last 15 min.

Postmortem. Not mandatory unless it recurs > 3 times / week.

RB-06 · `VllmTtftP95OutOfSlo` — TTFT P95 > 1.5 s for 5 min

Severity: CRITICAL (contractual SLO violation).

Immediate mitigation. Quick diagnosis of the regime (in order of likelihood):

If a v2 canary is active and the ratio ttft_p95(v2)/ttft_p95(v1) > 1.30: automatic rollback of the canary via Argo Rollouts (argo rollouts abort vllm-llama70b).
If num_requests_waiting > 5: scale out via KEDA.
If DRAM_ACTIVE > 90 % plus gpu_cache_usage_perc > 90 %: HBM bottleneck, reach for the quantisation lever or a context reduction.
If CLOCK_THROTTLE_REASONS != 0: see RB-02.

Evidence.

# Histogram snapshot
curl http://vllm-pod:8000/metrics | grep time_to_first_token
# Breakdown by version if a canary is running
# DCGM state at that moment
curl http://dcgm-exporter:9400/metrics | grep -E "PIPE_TENSOR|DRAM_ACTIVE|THROTTLE"
# Active traffic
kubectl top pods -n inference

Resolution. Depends on the diagnosis. Typical cases:

Canary regression → full rollback (see Canary).
Capacity saturation → scale replicas or accept a temporary 503 with Retry-After.
Prefill bound → enable or calibrate chunked prefill or disaggregated serving (see Disaggregated serving).

Closure. TTFT P95 inside SLO sustained 30 min.

Postmortem. Mandatory. Document the root cause and the lever applied; update the runbook.

Keep YAML workflows — three complete examples

Runbooks are only useful if they are codified in the workflow engine. Keep lets you declare them in YAML versioned in git.

Workflow 1 — `xid-detected.yaml`

workflow:
 id: xid-detected-drain
 name: "XID error detected — cordon node and capture evidence"
 description: "RB-03 implementation"
 triggers:
 - type: alert
 filters:
 - key: alertname
 value: GpuXidErrorDetected
 steps:
 - name: capture-evidence
 provider:
 type: bash
 with:
 command: |
 set -e
 NODE="{{ alert.labels.node }}"
 GPU="{{ alert.labels.gpu }}"
 INC_ID="{{ alert.fingerprint }}"
 mkdir -p /var/evidence/$INC_ID
 kubectl debug node/$NODE -it --image=nvcr.io/nvidia/cuda:12.4.0-base-ubuntu22.04 -- \
 bash -c "nvidia-smi -q -d ERROR,PCIE,ROW_REMAPPER > /host/var/evidence/$INC_ID/smi.txt"
 kubectl describe node $NODE > /var/evidence/$INC_ID/node.txt
 - name: cordon-node
 provider:
 type: kubernetes
 with:
 action: cordon
 name: "{{ alert.labels.node }}"
 if: "{{ alert.labels.severity == 'critical' }}"
 actions:
 - name: open-jira-ticket
 provider:
 type: jira
 config: "{{ providers.jira-prod }}"
 with:
 project: GPUOPS
 issuetype: Incident
 summary: "RB-03: XID {{ alert.annotations.xid_code }} on {{ alert.labels.node }}/{{ alert.labels.gpu }}"
 description: |
 Severity: {{ alert.labels.severity }}
 XID: {{ alert.annotations.xid_code }}
 Evidence: /var/evidence/{{ alert.fingerprint }}
 Runbook: https://runbooks.example.local/RB-03
 - name: notify-slack
 provider:
 type: slack
 config: "{{ providers.slack-gpu-incidents }}"
 with:
 message: |
 :warning: *RB-03 triggered*
 Node: `{{ alert.labels.node }}` GPU: `{{ alert.labels.gpu }}`
 XID: `{{ alert.annotations.xid_code }}`
 <{{ jira.url }}|Jira ticket>
 - name: emit-audit
 provider:
 type: kafka
 config: "{{ providers.kafka-audit }}"
 with:
 topic: audit.actions
 message:
 incident_id: "{{ alert.fingerprint }}"
 action: "cordon_node"
 actor: "keep-workflow"
 workflow_id: "xid-detected-drain"
 target: "{{ alert.labels.node }}"
 timestamp: "{{ now }}"

Workflow 2 — `ecc-dbe.yaml` — immediate paging

workflow:
 id: ecc-dbe-critical
 name: "ECC double-bit — page on-call and quarantine node"
 triggers:
 - type: alert
 filters:
 - key: alertname
 value: GpuEccDoubleBit
 steps:
 - name: cordon-immediately
 provider:
 type: kubernetes
 with:
 action: cordon
 name: "{{ alert.labels.node }}"
 - name: drain-workload
 provider:
 type: kubernetes
 with:
 action: drain
 name: "{{ alert.labels.node }}"
 options:
 ignore-daemonsets: true
 delete-emptydir-data: true
 grace-period: 120
 - name: page-oncall
 provider:
 type: pagerduty
 config: "{{ providers.pagerduty-critical }}"
 with:
 service_key: "{{ env.PD_SERVICE_KEY }}"
 severity: critical
 summary: "RB-04 ECC DBE on {{ alert.labels.node }}/{{ alert.labels.gpu }} — node drained"
 - name: emit-lifecycle
 provider:
 type: kafka
 config: "{{ providers.kafka-incidents }}"
 with:
 topic: incidents.lifecycle
 message:
 incident_id: "{{ alert.fingerprint }}"
 event: incident.opened
 severity: critical
 runbook: RB-04
 requires_postmortem: true
 - name: notify-dpo
 provider:
 type: email
 with:
 to: dpo@example.local
 subject: "ECC DBE on a production GPU — assessment required"
 body: |
 RB-04 ECC DBE incident detected on {{ alert.labels.node }}.
 Affected model: {{ alert.labels.model }}.
 Please assess whether personal/classified data was processed
 during the error window and whether GDPR art. 33 notification is needed.

Workflow 3 — `canary-rollback.yaml` — TTFT P95 out of SLO

workflow:
 id: canary-rollback-ttft
 name: "Rollback canary when TTFT P95 ratio v2/v1 > 1.30"
 triggers:
 - type: alert
 filters:
 - key: alertname
 value: VllmTtftP95OutOfSlo
 - key: canary_active
 value: "true"
 steps:
 - name: check-ratio
 provider:
 type: prometheus
 config: "{{ providers.prom-prod }}"
 with:
 query: |
 histogram_quantile(0.95, sum by(le)(rate(vllm:time_to_first_token_seconds_bucket{version="v2"}[5m])))
 /
 histogram_quantile(0.95, sum by(le)(rate(vllm:time_to_first_token_seconds_bucket{version="v1"}[5m])))
 condition: result > 1.30
 actions:
 - name: argo-rollback
 provider:
 type: kubernetes
 with:
 action: exec
 command:
 - kubectl
 - argo
 - rollouts
 - abort
 - "{{ alert.labels.rollout }}"
 - -n
 - "{{ alert.labels.namespace }}"
 - name: notify-and-audit
 provider:
 type: kafka
 config: "{{ providers.kafka-audit }}"
 with:
 topic: audit.actions
 message:
 incident_id: "{{ alert.fingerprint }}"
 action: canary_rollback
 ratio: "{{ steps.check-ratio.result }}"
 actor: keep-workflow
 timestamp: "{{ now }}"

Each workflow lives in repos/keep-workflows/ versioned in git, reviewed by pull request, validated by CI (keep workflow validate). The written runbook lives as docs/runbooks/RB-XX.md linked from the workflow YAML; the two always evolve together.

The canonical Kafka event schema

For the topics to be consumable by compliance, postmortem tooling and dashboards without each consumer having to guess the shape, the schema is fixed with Avro / Protobuf.

{
 "name": "IncidentLifecycleEvent",
 "type": "record",
 "fields": [
 { "name": "incident_id", "type": "string" },
 { "name": "event", "type": { "type": "enum", "symbols": [
 "incident.opened", "incident.acknowledged", "action.proposed",
 "action.executed", "action.failed", "incident.escalated",
 "incident.resolved", "postmortem.attached"
 ]}},
 { "name": "timestamp", "type": "string", "logicalType": "timestamp-millis" },
 { "name": "actor", "type": "string" },
 { "name": "severity", "type": { "type": "enum", "symbols": ["low","warning","critical"] } },
 { "name": "runbook", "type": ["null","string"], "default": null },
 { "name": "alert_name", "type": "string" },
 { "name": "labels", "type": { "type": "map", "values": "string" } },
 { "name": "annotations", "type": { "type": "map", "values": "string" } },
 { "name": "evidence_uri", "type": ["null","string"], "default": null },
 { "name": "requires_postmortem", "type": "boolean", "default": false }
 ]
}

For audit.actions (WORM), a separate and stricter schema with non-modifiable fields:

{
 "name": "AuditAction",
 "type": "record",
 "fields": [
 { "name": "incident_id", "type": "string" },
 { "name": "action", "type": "string" },
 { "name": "actor", "type": "string" },
 { "name": "actor_type", "type": { "type": "enum", "symbols": ["human","workflow","scheduler"] } },
 { "name": "workflow_id", "type": ["null","string"], "default": null },
 { "name": "target", "type": "string" },
 { "name": "command", "type": ["null","string"], "default": null },
 { "name": "result", "type": { "type": "enum", "symbols": ["success","failure","partial"] } },
 { "name": "timestamp", "type": "string", "logicalType": "timestamp-millis" },
 { "name": "evidence_uri", "type": ["null","string"], "default": null },
 { "name": "approver", "type": ["null","string"], "default": null }
 ]
}

The topic is configured with cleanup.policy=delete, retention.ms=15552000000 (6 months) and min.insync.replicas=2 with acks=all to guarantee durability. For longer retention without the Kafka cost, use tiered storage to Ceph RGW or an S3-compatible backend: new log in the hot tier, old log in the cold tier, transparently to the consumer.

Formal fit into incident management

Runbooks are not an isolated SRE practice. They fit into four regulatory frameworks that production LLM platforms touch daily.

ISO/IEC 27035 — information security incident management

It defines the formal cycle in five phases: plan & prepare → detect & report → assess & decide → respond → lessons learned. Each phase has outputs that must be documented. Translated to the stack:

Plan & prepare: runbooks RB-01 to RB-06 plus the Keep workflows are part of the Information Security Incident Management Plan. Versioned in git, reviewed annually.
Detect & report: the Prometheus alerts entering Kafka are the materialisation.
Assess & decide: the severity in gpu.alerts.enriched plus the Keep workflow logic.
Respond: execution of the workflow’s steps plus actions.
Lessons learned: a mandatory postmortem for the runbooks that flag it; the output documented in the postmortem repo plus the runbook update.

ENS (Esquema Nacional de Seguridad) — op.exp controls

op.exp.7 Incident management: the runbook catalogue plus the Keep / Kafka pipeline materialise the “organised and procedural response”.
op.exp.8 Activity logging: the audit.actions topic with WORM retention of 6 months (minimum for level ALTO).
op.exp.9 Incident management logging: the incidents.lifecycle topic with the full cycle of each incident.
op.exp.10 Protection of activity logs: WORM plus encryption at rest plus access control (compliance consumers read-only).

NIS2 — notification to the competent authority

For essential and important entities, art. 23 sets three deadlines from the detection of significant impact:

24 hours: an early warning to the national CSIRT (INCIBE-CERT in Spain).
72 hours: a formal notification with an initial assessment.
1 month: a final report with root cause, impact and corrective measures.

The data for those reports comes straight out of incidents.lifecycle plus audit.actions with a consumer that generates the dossier in the required format. Without the auditable pipeline, the NIS2 deadlines are unreachable.

EU AI Act — art. 73 (serious incident reporting)

Applicable to high-risk systems. Deadlines:

2 days: for incidents causing death or irreversible harm to people or critical infrastructure.
10 days: for incidents producing serious disruption of critical infrastructure.
15 days: for all other “serious incidents”.

The definition of “serious incident” includes systematic model failures, a fundamental rights breach, material or environmental harm. Runbooks must flag which alerts can lead to a serious incident (typically anything affecting the model’s output in a high-risk context) and fire a specific legal assessment sub-workflow.

ISO/IEC 42001 — AIMS clause 10 continual improvement

The mandatory post-incident postmortem feeds clause 10. Updating the runbook after every incident that reveals a new pattern is the “corrective action with verification of effectiveness” the standard demands. See ISO 42001 AIMS.

Four anti-patterns

Anti-pattern 1 — alerts without a runbook. The alert fires, the junior operator on call looks at the dashboard, searches Confluence, finds nothing up to date, calls the senior on Slack, waits 20 minutes. In that time the incident has grown. Rule: no alert goes to production without a published runbook and an approved Keep workflow. CI validates that every PrometheusRule with severity ≥ warning has its corresponding keep workflow.

Anti-pattern 2 — a runbook without prior evidence capture. The workflow runs nvidia-smi --gpu-reset as soon as the XID arrives, losing the state that would have diagnosed the root cause. The next identical XID forces the diagnosis to be redone from scratch. Rule: steps before actions; all evidence is captured first, destructive actions come after.

Anti-pattern 3 — escalation by seniority rota instead of severity. The junior operator on call handles an ECC DBE because “it is their turn”. They lack the context to understand row remap, retired pages or the data corruption risk. Rule: page by severity, not by rota: RB-04 and RB-03 page the senior primary ON-CALL with automatic escalation to infra/hardware if there is no acknowledgement within 10 min.

Anti-pattern 4 — no human gate for destructive actions. The workflow runs kubectl drain automatically on any alert marked CRITICAL. On the first false alarm (a transient that resolved itself in 30 s), Keep drained a production node during peak hour. Rule: destructive actions (drain, reset, RMA, full rollback) require human confirmation via a Slack interactive message, with a configurable timeout. A justified exception: ECC DBE confirmed by > 1 measurement, where the corruption risk outweighs the false alarm risk.

Applied to typical on-premise hardware

For a generic cluster of 4 nodes × 4×H100 SXM 80 GB with Kafka and Keep already deployed:

Kafka: a 3-broker cluster on non-GPU nodes of the K8s cluster; topics gpu.alerts.enriched, incidents.lifecycle, audit.actions configured with replication factor 3, min.insync.replicas 2. Audit with tiered storage to Ceph RGW for retention > 6 months without a brutal cost.
Keep: 2 operator replicas plus 1 worker replica in a keep namespace; connected to Prometheus (read provider), Kafka (read plus write provider), Slack, PagerDuty, Jira, Kubernetes (provider with a dedicated SA holding get/list/patch nodes and create jobs permissions).
Workflows: ~25-40 YAML files in the infra/keep-workflows/ repo, synchronised with the cluster via Flux or Argo CD. Validated by CI (keep workflow validate) on every PR.
Event volume: for 16 GPUs in normal operation with debounced alerts, ~50-200 events/day on gpu.alerts.enriched. In a typical incident, peaks of 500-2,000 events/day.
Compliance consumers: a python consumer in the compliance namespace that generates NIS2 / ENS / EU AI Act reports weekly, reading audit.actions and incidents.lifecycle.

What we have not covered (upcoming posts)

Postmortem playbooks — the mechanics of RCA with 5-whys, Ishikawa adapted to LLMs, integration with MLflow tracking for re-training if the postmortem produces an enriched dataset.
Chaos engineering for LLMs — controlled injection of XID errors, simulated ECC, artificial HBM latency to validate runbooks before the real incident.
Multi-cluster incident coordination — how to coordinate Keep across geographic clusters when an incident affects multiple regions.
CMDB and procurement integration — the RMA → ticket → ServiceNow → hardware replacement cycle automated via workflow.
LLM forensics — extracting the full OTel trace of a request affected by an incident, with redacted PII, kept in an evidence vault.

References

ISO/IEC 27035-1:2023 — Information security incident management — Principles and process.
ISO/IEC 27035-2:2023 — Information security incident management — Guidelines to plan and prepare for incident response.
ENS — Real Decreto 311/2022, Anexo II controles op.exp.7 a op.exp.10.
Directiva NIS2 (UE 2022/2555) — art. 23 (notificación de incidentes significativos).
Reglamento EU AI Act (UE 2024/1689) — art. 73 (reporting of serious incidents).
ISO/IEC 42001:2023 — AI management system — cláusula 10 (mejora continua).
Keep project — keephq.dev y github.com/keephq/keep (documentación de workflows YAML, providers).
Apache Kafka — Tiered Storage y cleanup.policy (docs.confluent.io / kafka.apache.org).
Confluent — Schema Registry y best practices para eventos lifecycle.
NVIDIA — Xid Errors Documentation y procedimientos de remediación.
Google SRE Book — Effective Troubleshooting y Postmortem Culture.
Atlassian — Incident Management Handbook (referencia para severity matrices).

ISO/IEC 42001: the operations manual for the AI system — how the AIMS fits over the on-premise LLM platform described in this blog

Mon, 01 Jun 2026 06:00:00 +0200

This post closes an asymmetry the blog had been accumulating: we have described in detail the technical platform (seven stack layers, seven deployment phases, five maturity levels), the operational pipeline (six LLMOps stages), the data pieces (corpus curation, versioning), the eval / safety pieces (evals, guardrails, LLM Guard) and the observe pieces (OTel GenAI tracing). What had not appeared is the governance layer a regulated client asks for on top of all that. ISO/IEC 42001 is that layer.

TL;DR

ISO/IEC 42001:2023 is the first certifiable international standard that defines how an AI system is managed. It is not a technical standard (it does not say “use this inference engine” or “this safety threshold”): it is a management standard, a cousin of ISO 27001 and ISO 9001. From both it inherits the Annex SL structure, seven mandatory clauses running through context, leadership, planning, support, operation, performance evaluation and improvement, and it adds an Annex A with 38 AI-specific controls in 9 sections: policies, internal organisation, resources, impact assessment, lifecycle, data, information to interested parties, use, third parties. The thesis of this post is that the technical architecture described in this blog directly covers between 60% and 80% of the A controls with no extra work. The LLMOps pipeline materialises A.6, versioning and curation materialise A.7, guardrails and evals materialise A.9, OTel tracing materialises A.8. The rest is governance discipline that does not appear in the code (a written AI policy, impact assessments per system, a stakeholder register, role decisions between provider/producer/customer, mandatory documentation), and that is precisely what separates a real certification from performative compliance. The post maps the correspondence control by control, cross-references it with the EU AI Act (seven articles directly aligned with 42001: 9, 10, 11, 12, 13, 14, 17), with NIS2 (asset register, incident notification, supply chain) and with ENS (Esquema Nacional de Seguridad, Spain’s national security framework, RD 311/2022, Basic/Medium/High categories), lists the seven mandatory minimum documents an auditor asks for, presents the blog’s multi-tenant chatbot as a live 42001 checklist, and closes with the five usual traps (confusing 42001 with EU AI Act compliance, over-documenting without measuring, ignoring A.5 until the audit, assuming 27001 covers the AI part, thinking certification is a one-off project rather than a living system).

The analogy: the aircraft operations manual

A modern aircraft, an A350, a Boeing 787, a drone certified for urban delivery, does not fly because it has good engines. It flies because the organisation that operates it has an Operations Manual approved by the aviation authority (EASA in Europe, the FAA in the US, AESA in Spain as a delegated body). The manual does not contain the engine blueprints, which the manufacturer certifies; it contains the procedures: who is captain on each flight, which checklist is run before each take-off, which periodic inspections happen at 100, 500 and 2,000 flight hours, which external suppliers are authorised to touch which components, what is documented after each incident, what to do when a new alert appears on the panel. The authority does not sit in on every flight: it reads the manual, randomly audits the traceability of past flights against the manual, and if everything adds up, it keeps the certification in place.

An AI system in production, the multi-tenant chatbot from the forensic post, a copilot for lawyers, a credit scoring system, is exactly the same. It flies because the model is good, the LLMOps pipeline is well built, the guardrails catch the bad cases. But it certifies because the organisation operating it has an AIMS (AI Management System) described in an auditable manual. ISO/IEC 42001 is that manual: its mandatory table of contents (Annex SL, seven clauses) and its catalogue of AI-specific controls (Annex A, 38 controls). The auditor does not sit next to the MLOps engineer: they read the AI policy, review the impact assessments of the most recently deployed systems, check that incident retrain is documented, verify third-party contracts, audit a sample of Langfuse traces cross-checked against dataset_hash and prompt_id. And if everything adds up, they certify.

The analogy matters because it frames the right question: 42001 does not certify the model or the code. It certifies the way of operating the complete system. A team can have the best OSS stack in the world and fail the audit because it has no written AI policy and no documented decision about which role (provider vs producer vs customer) it occupies towards its clients. And the reverse: a team with a modest model but with operations-manual discipline can certify without acrobatics.

ISO/IEC 42001 in 15 seconds

Publication: December 2023, ISO/IEC JTC 1/SC 42 (the ISO/IEC subcommittee for AI).
Status in 2026: an active standard, certifiable by accredited bodies (BSI, AENOR, TÜV, Bureau Veritas, A-LIGN, Schellman). Not yet formally recognised as a harmonised standard under the EU AI Act, but it provides the management base to lean on.
Compatibility: it shares the Annex SL structure with ISO 9001 (quality), 27001 (information security), 27701 (privacy), 22301 (continuity), 20000-1 (IT services). Organisations with integrated management systems (IMS) add it with 20-40% of the effort it would take to implement from scratch.
Applicability: any organisation that develops, provides, deploys or uses AI systems. It is not limited to developers: a company that consumes a hosted LLM and integrates it into its own product is within scope.
Certification: a 3-year cycle with an initial audit (Stage 1: documentation review + Stage 2: on-site audit) and annual surveillance audits. Typical cost: 15,000-60,000 € for the initial one depending on size; 6,000-20,000 € per annual surveillance.

What 42001 does not do:

It does not say which models to use or which thresholds to apply.
It does not certify the individual model (that is done by specific evaluations such as a NIST AI RMF profile or EU AI Act technical documentation).
It does not replace the EU AI Act or the GDPR: it is complementary. Implementing it well makes legal compliance easier but does not guarantee it.
It is not a technical standard for explainability or robustness (those are ISO/IEC 25059, 24029, 23894 and others in the SC 42 family).

Distinction from neighbouring frameworks

Framework	Nature	Scope	Certifiable	Overlap with 42001
ISO/IEC 42001:2023	Management standard	AIMS for any AI system	Yes	—
EU AI Act (Reg. 2024/1689)	Binding legal regulation	AI systems in the EU, risk-categorised	No (it is law)	Arts 9, 10, 11, 12, 13, 14, 17
NIS2 (Dir. 2022/2555)	Cybersecurity directive	Essential/important entities	Via the national scheme	Asset register, incident, supply chain
ENS (RD 311/2022)	Spanish security regulation	Public sector and its suppliers	Yes (B/M/A categories)	Traceability, incident management
ISO/IEC 27001	Management standard	Information security	Yes	Annex SL structure + Annex A overlap
ISO/IEC 27701	Management standard	Privacy (extends 27001)	Yes	PII in training data
NIST AI RMF 1.0	Voluntary framework	AI risk management	No	Conceptually aligned, not identical
ISO/IEC 23894	Technical standard	AI risk management	No	Input to A.5 (impact assessment)
ISO/IEC 5259	Family	Data quality for AI	No	Input to A.7 (data)

Three distinctions that matter operationally and are a recurring source of confusion with clients:

ISO 42001 ≠ EU AI Act compliance. Holding the 42001 certification makes it easier to demonstrate articles 9-17 of the European Regulation, but the Regulation requires things 42001 does not cover directly (CE marking of high-risk systems, registration in the European database, declaration of conformity, specific post-market monitoring). Implementing 42001 first and then filling the AI Act gaps is the standard route.
ISO 27001 is not enough. 27001 covers confidentiality, integrity and availability of information. The AI side is missing: bias, opacity, model drift, training corpus quality, human evaluation, impact on affected persons. 42001 is a complement, not a substitute. Organisations that already have 27001 in place have an advantage because they share half the documentation.
NIS2 ≠ AI safety. NIS2 requires registering critical assets, notifying incidents within 24 h, and managing the digital supply chain. AI systems can be within NIS2 scope if they form part of the critical asset (an LLM serving customer support in a financial entity is), but NIS2 does not audit model quality. 42001 does.

The seven clauses (Annex SL): the mandatory table of contents

The seven clauses, from clause 4 to clause 10, are common to all modern management standards (Annex SL, also called the “High Level Structure”). That means an organisation with ISO 9001 or 27001 already in place recognises the structure. Clauses 1-3 are introductory (scope, normative references, terms).

Clause 4 — Context of the organisation

Identify the external context (applicable regulation, customer expectations, social risks) and the internal context (strategy, culture, capabilities). Identify the interested parties and their expectations: customers, regulators, affected persons, employees, suppliers. Define the scope of the AIMS: which AI systems are in and which are out.

The usual gap: organisations that say “all our AI systems are in scope” without having enumerated them. The auditor asks for the list. Without a list, there is no scope.

Clause 5 — Leadership

Top management must approve and publish an AI policy, assign roles and responsibilities (typically AI lead, AI risk owner, data officer), and demonstrate commitment through resources, communication and oversight. The policy is an auditable document and must be made available to staff and interested parties.

The usual gap: a generic AI policy copied off the internet, with no measurables and no concrete objectives. The auditor asks how compliance with it is measured. Without metrics, the policy is theatre.

Clause 6 — Planning

Identify risks and opportunities of the AIMS (not of the individual model). Define measurable AI objectives, with deadlines and owners. Plan changes to the AIMS.

The usual gap: confusing AIMS risks (what happens if we do not document correctly?) with model risks (what happens if the model is biased?). The first belongs here; the second goes to A.5.

Clause 7 — Support

Resources: human, technical, financial, infrastructure. Competence of personnel (documented training). Awareness of the policy among personnel. Internal and external communication. Documented information (the backbone of the management system: policy, procedures, records, evidence).

The usual gap: documentation scattered across Confluence/Notion/Drive with no version control and no recorded approvals. The auditor asks about the last change: who approved it? when? on what grounds?

Clause 8 — Operation

The most operational clause. It requires:

Operational planning and control: how the AI system’s lifecycle is managed day to day. → Covered in the blog by the six-stage LLMOps pipeline.
Impact assessment (linked to A.5).
AI system lifecycle management (linked to A.6).
Data for AI systems (linked to A.7).

This is the clause that materialises in controls A.5, A.6 and A.7. On its own it adds no new requirements: it links to Annex A.

Clause 9 — Performance evaluation

Monitoring, measurement, analysis, evaluation. Internal audits (planned, with criteria, scope, frequency, recorded results). Management review (typically quarterly or half-yearly, with a mandatory agenda: inputs, evidence, decisions, actions).

The usual gap: there is OTel tracing + Langfuse + Grafana and plenty of data, but there is no formal management review agenda with documented minutes. The auditor asks for the minutes. Without minutes, there is no review.

Clause 10 — Improvement

Nonconformity and corrective action: when something fails, it is recorded, root cause is analysed, a correction is agreed, effectiveness is verified. Continual improvement: the system evolves deliberately.

The usual gap: Jira tickets with technical post-mortems but no formal “ISO nonconformity” record closed out with an effectiveness check. They are two distinct artefacts, even if they can be integrated.

The 38 Annex A controls: the AI-specific catalogue

Unlike Annex SL (which is common), Annex A is 42001’s AI-specific signature. The 38 controls are organised into 9 sections (A.2 to A.10; A.1 is the introduction) covering the AI-specific risks: opacity, bias, drift, corpus quality, impact on affected persons, third-party dependency. Each control has an objective (what it aims to achieve) and implementation guidance in Annex B.

Section	Focus	# controls
A.2	AI-related policies	2
A.3	Internal organisation	3
A.4	Resources for AI systems	6
A.5	Impact assessment	5
A.6	AI system lifecycle	4
A.7	Data for AI systems	5
A.8	Information for interested parties	4
A.9	Use of AI systems	3
A.10	Third parties and customer relationships	4
Total	—	38

What follows is the section-by-section mapping of the controls onto material we have already covered in the blog. The editorial intent is to show which gaps remain once the technical architecture is in place, so that the road to certification does not start from zero.

Cross-mapping: 38 controls ↔ blog posts

A.2 — AI policies (2 controls): PARTIAL

A.2.2 AI policy: the organisation must have a documented AI policy, approved by management, reviewed periodically, communicated and available. It covers principles, scope and commitments.
A.2.3 Alignment with other policies: the AI policy is not an orphan, it aligns with existing security, privacy, quality and ethics policies.

Gap: this is not a code matter. The AI policy is a document that the organisation’s management approves and signs. The blog teaches a neutral, technical editorial stance (no hype, sovereignty, OSS by default under ENS/NIS2), but that is not any specific organisation’s AI policy. Each client has to draft and sign their own.

Minimum template: 1-2 pages with: principles (transparency, human oversight, fairness, accountability, sustainability), scope (which systems), measurable commitments (annual review, impact assessment before deployment, team training), governance (who approves what).

A.3 — Internal organisation (3 controls): GAP

A.3.2 AI roles and responsibilities: roles defined, non-overlapping, communicated. Typically: AI lead, AI risk owner, data steward, AI ethics officer (which may be a single person in small organisations).
A.3.3 Reporting of AI incidents/concerns: a channel for anyone (internal or external) to report a problem with an AI system, with documented follow-up.
A.3.4 Identification of stakeholders: a maintained list of stakeholders (customers, affected persons, regulators, partners) and their expectations.

Gap: also not technical. An organisational decision. The usual way to cover it is to appoint an AI lead (which may be the CIO, the CTO or a new role depending on size), reuse the security reporting channel (typically already in place because of 27001) by extending it to AI, and maintain a live stakeholder register.

A.4 — Resources (6 controls): COVERED

A.4.2 Documented information: documentation of the AIMS.
A.4.3 Data resources: identification and management of the data available for training, evaluation and operation.
A.4.4 Tooling resources: development, validation and monitoring tools.
A.4.5 System resources: hardware, infrastructure, compute.
A.4.6 Human resources: competent personnel.
A.4.7 Financial resources: budget.

Covered by the blog in the three architectural posts:

Anatomy of the stack: seven layers: A.4.5 system resources, A.4.4 tooling.
Seven deployment phases: A.4.5 + A.4.7 (budget implicitly).
Five maturity levels: A.4.5 + A.4.6 (team maturity).
OSS tooling catalogue: A.4.4 tooling.
Data versioning with DVC and lakeFS: A.4.3 data resources.

A.5 — Impact assessment (5 controls): PARTIAL

A.5.2 AI impact assessment process: a documented impact assessment procedure.
A.5.3 Documentation of AI impact assessments: records of the assessments carried out.
A.5.4 Alignment with AI risk treatment: the impact assessment’s decisions feed risk treatment.
A.5.5 Impacts on individuals: specific dimensions covering affected persons (rights, discrimination, privacy).
A.5.6 Societal impacts: dimensions covering society (information, social rights).

Partial: the method exists in the ISO/IEC SC 42 family. ISO/IEC 23894:2023 is the technical standard for AI risk management and NIST AI RMF 1.0 is the freely usable American equivalent. But the organisation has to write its procedure and run it per system before deployment. It is not code, it is discipline.

Minimum template for the impact assessment (3-5 pages per system):

Description of the system (what it does, who it serves, underlying model and stack).
Stakeholders identified.
Potential impacts (intended and unintended) on people, groups and society.
Fairness and robustness metrics applied, with thresholds and results.
Mitigations applied (guardrails, evals, human oversight, rate limiting).
Accepted residual risks, with signed justification.
Review cadence (typically annual, or on substantial change).

A.6 — AI system lifecycle (4 controls): COVERED

A.6.2.2 Objectives for responsible development of AI: responsible development objectives defined per system.
A.6.2.3 Processes for responsible AI design and development: design and development procedures.
A.6.2.4 AI system requirements and specifications: formal specification of the system.
A.6.2.5 Verification and validation: V&V before and during operation.
A.6.2.6 Deployment: deployment procedures.
A.6.2.7 Operation and monitoring: continuous operation and monitoring.
A.6.2.8 Documentation: lifecycle documentation.

Covered by the blog:

The six-stage LLMOps pipeline: the complete master map of the lifecycle.
Anatomy of an LLM request: the forensic version of how it runs in production.
Continuous fine-tuning in production: the A.6.2.3 + A.6.2.5 + A.6.2.6 + A.6.2.7 discipline in real operation.
Evals: the layer after tracing: A.6.2.5 verification and validation.
Retrain: A.6.2.7 operation + continuous iteration.
Modern alignment: DPO, KTO, ORPO, SimPO: A.6.2.3 responsible design.
MLOps landscape 2026: the tooling landscape.

A.7 — Data for AI systems (5 controls): COVERED

A.7.2 Data for development and enhancement of AI: data management policy and procedures for development and improvement.
A.7.3 Acquisition of data: acquisition procedures (origin, authorisation, quality).
A.7.4 Quality of data for AI systems: measurable quality criteria.
A.7.5 Data provenance: data lineage.
A.7.6 Data preparation: preparation procedures (chunking, anonymisation, labelling).

Covered by the blog:

RAG corpus curation: the active librarian: A.7.4 + A.7.5 + A.7.6 in detail (five layers: schema, dedup, PII, anti-contamination, lineage).
Data versioning: DVC and lakeFS: A.7.2 + A.7.5 (the four data artefacts versioned with lineage).
Reranker and hybrid retrieval: A.7.6 preparation + filtering.
LLM Guard: A.7.6 runtime anonymisation with Vault.

A.8 — Information for interested parties (4 controls): COVERED

A.8.2 System documentation and information for users: technical documentation available.
A.8.3 External reporting: the capacity to report to authorities where applicable.
A.8.4 Communication of incidents to users: notification to users when there is an incident.
A.8.5 Information for interested parties: information for other stakeholders.

Covered by the blog:

LLM tracing with OpenTelemetry GenAI: A.8.2 per-request traceability, A.8.3 the capacity to extract forensic reporting.
Prompt versioning with Langfuse and MLflow: A.8.2 documented versioning.
Guardrails and safety in LLMs: A.8.4, gen_ai.guardrail.* spans as the basis for incident notification.
LLM Guard: A.8.4 incident events for retrain.

A.9 — Use of AI systems (3 controls): COVERED

A.9.2 Processes for responsible use of AI: responsible use procedures.
A.9.3 Objectives for responsible use of AI: objectives.
A.9.4 Intended use of AI systems: documentation of the intended use.

Covered by the blog:

Guardrails and safety in LLMs: A.9.2 + A.9.3 (the four lines of defence).
LLM Guard: A.9.2 operational detail.
Evals: the layer after tracing: A.9.3 measurement of objectives.
Retrain: A.9.2 closed loop.

A.10 — Third parties and customer relationships (4 controls): COVERED

A.10.2 Allocation of responsibilities: distribution of responsibilities between AI roles.
A.10.3 Suppliers: procedures for AI suppliers.
A.10.4 Customers: procedures towards customers.
A.10.5 Third parties: procedures for third parties.

Covered by the blog:

The parallel catalogue: OSS vs hyperscalers: A.10.3 supplier assessment with lock-in and contractual sovereignty analysis.
The OSS catalogue for LLMOps: A.10.5 third-party inventory (OSS components with licences and governance).
Anatomy of an LLM request: A.10.2 + A.10.4 in the multi-tenant case.

The roles defined by the standard

ISO/IEC 22989:2022 (AI vocabulary, complementary to 42001) defines six roles. Each organisation has to decide which ones it occupies and document it:

Role	Definition	Main responsibility	Example
AI provider	Organisation that provides the AI system to others	Makes the system available	OpenAI provides GPT-5 via API
AI producer	Organisation that develops the AI system	Design, development, validation	Meta produces Llama 4
AI customer	Organisation that acquires the AI system	Selection, integration, oversight	A consultancy integrating an LLM into its own product
AI partner	Organisation collaborating with another AI role	Shared	A GPU hardware manufacturer
AI subject	Person or group affected by the system	Recipient of the impact	The chatbot’s end user
Relevant authority	Regulator with jurisdiction	External oversight	AEPD, CNMC, EU AI Act authorities

An organisation can occupy several roles at once, which changes the applicable controls. A common pattern in consultancy is: producer + customer + provider towards the end client. The A.10 responsibilities are modulated by the roles.

Example role mapping for the multi-tenant chatbot from the forensic post:

Base model manufacturer (Llama 4): AI producer of the base model.
OSS stack operator (the consultancy): AI producer of the LoRA adapter + AI provider of the chatbot to its clients + AI customer of Meta’s base model.
End client (the insurer): AI customer of the chatbot + AI provider of the customer support service.
The policyholder: AI subject.
AEPD + EU AI Act authority: relevant authority.

Each box in the table generates different obligations. The consultancy, being producer of the adapter, must document A.6 (lifecycle) and A.7 (data) for the adapter. Being provider of the chatbot, it must document A.10.4 (customers). Being customer of the base model, it must document A.10.3 (suppliers) and validate that Meta does its part.

Impact levels and proportionality

42001 does not demand the same rigour of every system. Clause 6.1.2 and control A.5 introduce impact as a modulator. The standard does not define strict categories (unlike the EU AI Act, which does define “prohibited / high risk / limited risk / minimal”), but it recommends using levels according to severity and probability.

Industry practice in 2026 aligns the 42001 levels with the EU AI Act categories:

42001 level	EU AI Act	Examples	Control depth
High	High risk (Annex III)	Credit scoring, HR, health, critical infrastructure	Exhaustive impact assessment, mandatory human oversight, continuous monitoring, adversarial evals, detailed logging, half-yearly management review
Medium	Limited risk	Customer service chatbots that do not automate decisions, productivity assistants	Standard impact assessment, complete guardrails, annual review
Low	Minimal risk	Spam filters, non-personalised content recommendations	Lightweight impact assessment, basic controls

This proportionality is operationally key: implementing 42001 at maximum rigour for a low-risk system is waste; relaxing it on a high-risk one is non-compliance.

The seven minimum AIMS documents

An auditor at Stage 1 (documentation review) asks for between seven and ten documents. The seven essentials:

AI policy (clause 5.2 + A.2.2). 1-2 pages. Approved by management, dated, versioned.
AIMS scope (clause 4.3). List of AI systems in scope, inclusion criteria.
Stakeholder register (clause 4.2 + A.3.4). A maintained list with expectations.
AIMS risk register (clause 6.1). Risks of the management system, not of each model.
Impact assessment procedure (A.5.2) + records of assessments carried out (A.5.3). The procedure plus one or more assessments done.
AI lifecycle procedure (A.6.2), which can literally read “see the six-stage LLMOps pipeline” with references to technical runbooks.
Data management procedure (A.7.2), covering acquisition, quality, provenance, preparation, anonymisation.

Common additional documents:

Responsible use policy (A.9.2) with permitted and non-permitted use types.
AI third-party management procedure (A.10.3, A.10.5) with assessment criteria for AI suppliers.
Internal audit plan + management review agenda (clauses 9.2 + 9.3).

For an organisation with a mature OSS stack, documents 6 and 7 are references to technical artefacts that already exist (pipeline runbooks, DVC configurations, the PII policy in LLM Guard). The real documentary effort sits in documents 1, 2, 3, 4 and 5.

Applied case: the blog’s multi-tenant chatbot → 42001 checklist

We take the system described in the forensic post, a multi-tenant customer support chatbot for insurers on an on-premise OSS stack, and walk it as a 42001 auditor would.

Clause 4 — Context. The AIMS scope includes the chatbot and excludes the internal HR system (a different AI). Stakeholders identified: client insurers, affected policyholders, the AEPD, the EU AI Act authority (once it enters into force on 2 Aug 2026), the supplier Meta (base model), the hardware supplier NVIDIA. → Documented.

Clause 5 — Leadership. AI policy signed by the CEO, in force. Roles assigned: AI lead (CTO), AI risk owner (CISO), data steward (Head of Data), quarterly AI ethics committee. → Documented.

Clause 6 — Planning. AIMS risk register: incomplete documentation, team churn, dependency on a single GPU supplier, EU AI Act regulatory change. AIMS objectives for 2026: 42001 certification before Q4, EU AI Act high-risk compliance before 2 Aug. → Documented.

Clause 7 — Support. Resources: a 4×H100 SXM cluster + the seven stack layers. Competence: 2 MLE + 2 SRE + 1 part-time AI ethics, all with documented training. Communication: AI policy on the intranet + handbook. → Documented.

Clause 8 — Operation. Operating procedures = the six-stage LLMOps pipeline. Impact assessment run before deployment + annual review + review on substantial change (defined as: base model change, major adapter change, expansion to a new tenant). → Documented.

Clause 9 — Evaluation. Monitoring: Langfuse + Tempo + VictoriaMetrics + Grafana. Mandatory dashboard metrics: F1 per guardrail category over real traffic, statistical drift, RAG faithfulness, refusal rate. Quarterly internal audit with written criteria. Half-yearly management review with signed minutes. → Documented.

Clause 10 — Improvement. Incident-driven retrain tickets mapped as nonconformities when severity ≥ HIGH. Documented root cause analysis. Effectiveness verified at the next eval gate. → Documented.

Annex A — By section:

A.2 (Policies): AI policy + responsible use policy. → Documented.
A.3 (Organisation): roles assigned, reporting channel, stakeholder register. → Documented.
A.4 (Resources): seven deployment phases + OSS catalogue + training plan + annual budget. → Documented.
A.5 (Impact): procedure + per-system assessments + fairness metrics applied. → Documented.
A.6 (Lifecycle): LLMOps pipeline + continuous fine-tuning + retrain. → Documented.
A.7 (Data): data versioning + RAG corpus curation + LLM Guard Vault + Presidio. → Documented.
A.8 (Information to parties): OTel tracing + Langfuse + gen_ai.guardrail.* spans + notification to tenants within SLA. → Documented.
A.9 (Use): guardrails + evals + responsible use policy. → Documented.
A.10 (Third parties): OSS vs hyperscalers with lock-in analysis + Meta contract for the base model + contracts with tenants. → Documented.

Result of the walkthrough: certifiable. The typical gaps (A.2.2 written policy, A.3 roles, A.5 impact assessment procedure) are covered as formal documents. The operational clauses (8, 9, 10) lean on the blog’s technical architecture. The distance between “having the architecture” and “having certification” is measured in documentary discipline, not in code.

Cross-mapping with the EU AI Act, NIS2 and ENS

EU AI Act (Reg. 2024/1689) — seven directly aligned articles

EU AI Act article	Topic	Aligned 42001 control	Applies to
Art. 9	Risk management system	A.5 + clause 6	High-risk systems
Art. 10	Data and data governance	A.7 (all)	High-risk systems
Art. 11	Technical documentation	A.6 + A.4.2	High-risk systems
Art. 12	Record-keeping (logs)	A.8.2 + OTel tracing	High-risk systems
Art. 13	Transparency to deployers	A.8.5 + A.10.4	High-risk systems
Art. 14	Human oversight	A.9.2 + documented oversight	High-risk systems
Art. 17	Quality management system	Clauses 4-10	High-risk providers

The main obligations for high-risk systems enter into application on 2 August 2026. Implementing 42001 now builds the management base that deadline demands.

What is missing for EU AI Act compliance that 42001 does not cover:

CE conformity of high-risk systems (declaration of conformity, marking, registration in the EU database).
The specific post-market monitoring required by Art. 72.
Serious incident reporting to authorities within legal deadlines (not only to users).
Transparency obligations towards users for limited-risk systems (Art. 50): chatbots, deepfakes, generated content.
The Art. 5 prohibitions (social scoring, manipulation, real-time biometrics with exceptions).

NIS2 (Dir. 2022/2555) — three pillars with overlap

Asset register (Art. 21.2.f): AI systems within NIS2 scope must be in the asset inventory. → Overlaps with A.4 + clause 4.3 (scope).
Incident notification (Art. 23): significant incidents are notified within 24 h (initial alert) + 72 h (detailed report). → Overlaps with A.3.3 (reporting) + clause 10 (improvement).
Supply chain security (Art. 21.2.d): security assessment of the digital supply chain. → Overlaps with A.10.3 (suppliers).

For essential NIS2 entities that also use AI systems, 42001 covers the AI-specific part that NIS2 requires by inference but does not spell out.

ENS (RD 311/2022)

The Spanish Esquema Nacional de Seguridad already expressly addresses AI in its Annex II (ENS controls). The Basic/Medium/High categories align with the 42001 impact levels. The ENS controls on traceability (op.exp.8), activity logging (op.exp.10) and incident management (op.exp.7) are covered by the same technical artefacts as A.8 and A.5 of 42001. An organisation certified at ENS High category with AI systems is a reasonable effort away from adding 42001.

The five usual certification traps

Trap 1 — Confusing 42001 with EU AI Act compliance. Passing the 42001 audit does not imply conformity with the European Regulation. They are distinct universes with a 60-70% overlap. The trap surfaces when the client asks for CE marking of the high-risk system and the organisation produces only the 42001 certificate.

Trap 2 — Over-documenting. Two-hundred-page manuals with procedures copied from templates, with no measurables and no evidence of application. The auditor asks for the last execution of the procedure; with no records, the procedures are ornament. The practical rule: prefer short documents referencing live technical artefacts to long self-contained ones.

Trap 3 — Under-measuring. Defining AIMS objectives with no operational metrics. “Improve model quality” is a null objective; “F1 per guardrail category ≥ 0.85 over real traffic, measured weekly, reviewed quarterly in the management review” is an auditable objective. The blog has insisted on this in every post on evals, guardrails and retrain.

Trap 4 — Ignoring A.5 until audit day. The impact assessment is the most underrated control and the first one the auditor asks for. Without per-system assessments carried out before deployment, there is no way to demonstrate A.5. The trap surfaces when there is no longer time to produce credible retrospective assessments.

Trap 5 — Assuming 27001 covers the AI part. Organisations that already have 27001 sometimes think “we have half of it done”. That is true for Annex SL (structure) and for A.5/A.6/A.7 of 27001 (not of 42001) as far as infosec goes. It is false for A.5 of 42001 (impact assessment), A.7 of 42001 (AI-specific data quality), A.9 (responsible use) and A.10.4 (AI customers). You have to add, not assume.

What we have not covered (upcoming posts)

Concrete templates of the seven mandatory documents, with drafting examples and metrics. Material for a post along the lines of “The AIMS manual in 7 documents” with a reference frame.
Detailed article-by-article mapping to the EU AI Act with the checklist of technical evidence derivable from the blog’s OSS stack. Especially Arts 11 (technical documentation), 14 (human oversight) and 72 (post-market monitoring).
The combined ENS High category + 42001 case: which ENS controls are covered by which AIMS artefacts, avoiding duplication.
NIST AI RMF 1.0 vs 42001 comparison: many international clients ask for both. How the same artefacts are recycled to satisfy the two frameworks.
42001 for LLM agents and MCP: the new dimensions that emerge when the AI system is agentic (excessive agency, tool use, graduated autonomy). The guardrails post introduced line 3 (tool GR); 42001 has open gaps in this terrain and SC 42 is working on addenda.

References

ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system. ISO. https://www.iso.org/standard/81230.html.
ISO/IEC 22989:2022 — Information technology — Artificial intelligence — Artificial intelligence concepts and terminology. Define los roles AI provider/producer/customer/partner/subject.
ISO/IEC 23894:2023 — Information technology — Artificial intelligence — Guidance on risk management. Insumo de A.5.
ISO/IEC 38507:2022 — Governance implications of the use of AI by organizations. Complemento de gobierno.
ISO/IEC 5259 — Data quality for analytics and machine learning (familia). Insumo de A.7.
EU AI Act (Regulation 2024/1689) — texto consolidado en EUR-Lex. Entrada en vigor de obligaciones de alto riesgo: 2 ago 2026.
NIS2 (Directive 2022/2555) — texto consolidado en EUR-Lex.
ENS — Real Decreto 311/2022 — Esquema Nacional de Seguridad, BOE-A-2022-7191.
NIST AI RMF 1.0 (2023) — https://www.nist.gov/itl/ai-risk-management-framework.
EUR-Lex EU AI Act consolidated text — https://eur-lex.europa.eu/eli/reg/2024/1689.
A-LIGN / BSI / Schellman — blogs sobre experiencia de auditoría 42001 con casos reales 2024-2025.

EU AI Act: the technical dossier, article by article, over the blog's on-premise LLM architecture

Mon, 01 Jun 2026 05:30:00 +0200

Sister post to the mapping onto ISO/IEC 42001. That one broke down the AI management system, the certifiable standard. This one breaks down the binding legal regulation that applies without any certification: the EU AI Act is direct law in all 27 Member States, with no transposition and with explicit penalties of up to 35 million euros or 7% of worldwide turnover. The main obligations for high-risk systems enter into force on 2 August 2026; each article applies from its own date, not from the date the organisation gets certified.

TL;DR

Regulation (EU) 2024/1689 (the EU AI Act, “Reglamento Europeo de Inteligencia Artificial”), published in the Official Journal on 12 July 2024, sets obligations by risk tier (prohibited, high, limited, minimal) and by role (provider, deployer, importer, distributor, authorised representative). The obligations for high-risk systems (Annex III: biometrics, critical infrastructure, education, employment, essential public and private services, law enforcement, migration, justice, democratic processes) enter into force on 2 August 2026 and are the category that applies to most LLM projects in a medium-to-large enterprise. This post maps, article by article, the obligations relevant to a high-risk LLM system deployed on-premise: each article states its requirement, identifies which blog post already describes the technical piece that materialises it, and closes with an auditable checklist that a provider hands to a national supervisory authority. The Annex IV technical dossier is reconstructed by pointing its nine mandatory sections at the corresponding technical runbooks. Also covered: the Art. 6 classification and how to decide whether a system falls in as high risk, the Art. 5 prohibitions (what is excluded by construction), the GPAI obligations of Art. 53 that affect anyone building on base models (Llama, Mistral, DeepSeek, Qwen) rather than their own, the full timetable of application dates (5 Aug 2024 entry into force, 2 Feb 2025 prohibitions, 2 Aug 2025 GPAI, 2 Aug 2026 high risk Annex III, 2 Aug 2027 high risk Annex I and systemic GPAI), the Art. 99 penalty table (up to 35 M€ or 7% of worldwide turnover for Art. 5 violations, 15 M€ or 3% for high risk, 7.5 M€ or 1% for incorrect information) and the five frequent compliance traps. The editorial thesis: the technical architecture described in this blog directly covers between 70% and 85% of the Regulation’s technical requirements; the rest is documentary and procedural discipline (FRIA, CE marking, signed declaration of conformity, EU database registration, incident reporting within legal deadlines) built on top of technical artefacts that already exist but running on its own circuit.

The analogy: the type-approval dossier for a new vehicle

A vehicle manufacturer cannot sell a new car in the EU without passing WVTA (Whole Vehicle Type Approval). The process is public and standardised: the manufacturer prepares a technical dossier with dozens of chapters (brakes, emissions, active and passive safety, lighting, noise, weight, dimensions, recyclable materials, driver assistance devices), submits it to a type-approval authority or a notified technical service, which audits the file and, if everything adds up, issues the type approval. The manufacturer then stamps the E mark (E1 Germany, E9 Spain, and so on) and the CE/UNECE plate on every vehicle of that type produced in series. Each vehicle also carries a service book with mandatory inspections, and serious accidents or systemic defects are reported to the competent ministry, which can order recalls.

The EU AI Act adapts exactly this industrial model to high-risk AI software. The provider prepares the Annex IV dossier (nine mandatory sections) with the full technical documentation of the system, runs a conformity assessment (self-assessment for most Annex III cases, notified body for the more sensitive Annex I ones), signs the EU declaration of conformity, registers the system in the European database, and applies the CE marking to the system when it is placed on the market. From there on it must keep a live post-market monitoring system and report serious incidents to the market surveillance authorities of each Member State within legal deadlines ranging from 2 to 15 days depending on severity. Failing to comply, penalties reach 35 million euros or 7% of worldwide turnover depending on which article was breached.

The analogy matters because it bounds expectations: this is not a one-off compliance job, nor a badge you buy. It is an industrial type-approval process, with cadences, evidence, signatures and criminal liability. And most of the technical evidence the dossier asks for already exists in any serious system described in this blog: data lineage, OTel tracing, continuous evals, guardrails, incident-driven retrain, a responsible-use policy. What is missing is assembling it in the correct legal format.

How it fits with ISO 42001, NIS2 and ENS

Before dropping into the articles, a reminder of the editorial position from the post on ISO 42001:

Piece	Nature	Who operates it	Coverage
EU AI Act	Direct EU law	Provider + deployer + national surveillance authorities	AI systems on the EU market, segmented by risk
ISO/IEC 42001	Certifiable management standard	Organisation + certification body	AIMS, organisational governance
NIS2	Transposed cyber directive	Essential/important entities	Asset register, incident notification, supply chain
ENS (Esquema Nacional de Seguridad, Spain’s national security framework, RD 311/2022)	Spanish security regulation	Public sector + its suppliers	Categories B/M/A, certifiable

Implementing 42001 makes it easier to demonstrate articles 9-17 of the EU AI Act, but it is not equivalent to legal compliance. The table of legal obligations and the chain of criminal liability come from the Regulation, not from the standard. An organisation certified against 42001 that deploys a high-risk system without CE marking, without registration in the EU database, without a signed declaration of conformity and without a documented FRIA breaches the Regulation even with the certificate on the wall.

The four risk categories and the Art. 6 classification

The Regulation classifies systems into four risk tiers. The choice of category belongs to the provider and must be documented in the dossier, with technical and legal justification.

Category	Article	Examples	Consequence
Prohibited	Art. 5	Social scoring, manipulation, real-time biometrics with exceptions, indiscriminate facial scraping	Cannot operate in the EU under any circumstances
High risk	Art. 6 + Annex I + Annex III	Credit scoring, HR, education, critical infrastructure, non-real-time biometrics, justice, migration, health	Complies with Arts. 9-17, Annex IV dossier, CE marking, EU DB registration
Limited risk	Art. 50	Chatbots with humans as users, deepfakes, synthetic content	Transparency obligations towards the user
Minimal risk	Everything else	Spam filters, video game NPCs, content suggestions	No specific obligations; a voluntary code of conduct is recommended

The Art. 6 test for deciding whether a system is high risk:

Is it in Annex I? (products regulated by the listed harmonisation legislation: machinery, lifts, toys, medical devices, and so on). If the AI system is a safety component of an Annex I product, it is high risk.
Is it in Annex III? (eight areas: biometrics, critical infrastructure, education, employment, essential public/private services, law enforcement, migration/asylum, justice/democratic processes). If the system falls in any of those areas it is high risk, except if the Art. 6.3 exception is invoked (systems performing a narrow procedural task, improving the result of a previous human activity, detecting decision patterns without influencing the final decision, preparatory tasks).

The Art. 6.3 exception requires formal documentation justifying why it does not apply. In other words, even falling outside is not free: you have to demonstrate why.

For the typical LLM systems covered in this blog:

Customer support chatbot for banking, insurance or healthcare: probably high risk if it automates contractual decisions about the customer, limited risk if it only informs.
Internal HR assistant (CV screening): high risk (Annex III, employment area).
Medical assistant (diagnostic support): high risk (Annex III, healthcare services area).
Fraud detection system: high risk (Annex III, financial services area if it affects access to credit).
Code copilot for developers: minimal risk (does not affect third parties’ fundamental rights).
Internal LLM as a service with no productive use: minimal risk.

The category decision is not an opinion: it is documented and justified in the dossier.

Timetable of application

The obligations arrive in stages. The dates are inflexible:

Date	Applies
1 Aug 2024	The Regulation enters into force
2 Feb 2025	Art. 5 prohibitions + Art. 4 AI literacy obligations
2 Aug 2025	GPAI obligations (Art. 53) + governance + general penalties + designated national authorities
2 Aug 2026	Main high-risk obligations for Annex III + Art. 50 transparency to users
2 Aug 2027	High risk under Annex I (components of regulated products) + GPAI with systemic risk

2 August 2026 is the date that matters to most enterprise LLM projects. We are in June 2026: less than two months to go.

Article-by-article mapping

The sections below follow the structure of the Regulation. For each article the requirement is stated, the blog’s technical artefact that covers it is identified, and it closes with an auditable checklist.

Art. 5 — Prohibited practices (in force since 2 Feb 2025)

What it requires. It prohibits placing on the market, putting into service or using AI systems that:

Manipulate behaviour through subliminal or deceptive techniques that distort decision-making.
Exploit vulnerabilities related to age, disability or socio-economic situation.
Implement social scoring by public authorities.
Perform individual predictive policing based on profiling.
Perform indiscriminate facial scraping to build facial recognition databases.
Infer emotions in workplaces or education, except for medical or safety reasons.
Perform biometric categorisation inferring sensitive attributes (race, political opinion, sexual orientation, and so on).
Real-time remote biometric identification in public spaces by law enforcement, with strict exceptions (terrorism, kidnapping, missing persons) subject to prior judicial authorisation.

The blog’s stack. No piece of the blog facilitates these practices; the OSS catalogue described is oriented towards legitimate tasks. But the provider must explicitly document why the system does not fall under these prohibitions. It cannot be assumed.

Checklist.

Prohibition analysis documented per system, with a written statement of non-applicability.
If the system uses facial biometrics or emotion analysis: specific legal analysis with a formal opinion.
Annual legal review, or on any change of functionality.

Art. 6 — Classification of high-risk systems

What it requires. Determine whether the system is high risk by being in Annex I (safety component of a regulated product) or Annex III (8 areas). If it is in Annex III, assess the Art. 6.3 exception where applicable.

The blog’s stack. The forensic post and the six-stage pipeline describe systems that are typically high risk (multi-tenant chatbot affecting service decisions in a regulated sector).

Checklist.

Art. 6 analysis signed by the organisation’s legal officer.
If Art. 6.3 is invoked: formal documentation of the exception.
Re-assessment if the functionality changes (for example, the assistant moves from informing to deciding).

Art. 9 — Risk management system

What it requires. An iterative system, planned and executed across the whole lifecycle. Identify foreseeable risks, estimate risks under normal use and foreseeable misuse, evaluate emerging risks in post-market monitoring, adopt mitigation measures, communicate residual risks. Effectiveness testing is done under realistic conditions.

The blog’s stack.

The six-stage LLMOps pipeline, the iterative lifecycle.
Evals: the layer after tracing, testing under realistic conditions with golden sets.
Guardrails and safety in LLMs, operational mitigation measures.
Retrain: closing the loop, management of emerging risks via incident-driven retrain.

Checklist.

Risk management document per system, with identification, mitigation, and accepted residual risks signed off.
Periodic review procedure (at least annual, or on substantial change).
Link to the documented improvement loop.

Art. 10 — Data and data governance

What it requires. Training, validation and testing datasets that are relevant, representative, as free of errors and as complete as possible, taking into account the characteristics of the intended purpose. Document:

Data collection and selection.
Processing and annotation.
Biases identified as likely to affect fundamental rights or cause discrimination; measures to prevent them.
Identification of data gaps and how they are addressed.

The blog’s stack.

Data versioning with DVC and lakeFS, the four data artefacts plus end-to-end lineage.
RAG corpus curation: the active librarian, five layers: schema, dedup, PII, anti-contamination, lineage.
LLM Guard: Vault and Anonymize, runtime anonymisation with restitution.

Checklist.

Data governance document per dataset (training / RAG corpus / golden eval / enriched retrain).
Bias analysis by protected category with metrics (parity ratio, equalized odds, calibration).
PII / anonymisation / pseudonymisation procedure with measured F1.
Justification of representativeness for the intended context.
Verifiable chunk→trace lineage.

Art. 11 + Annex IV — Technical documentation

What it requires. A technical dossier with the nine Annex IV sections, drawn up before the system is placed on the market, maintained during operation, and available to authorities for ten years after the last operation. The nine sections:

General description of the system: name, version, purpose, integrator, intended hardware, instructions for use.
Detailed description of design and development: architecture, base models, training methods, design decisions with justification.
Information on monitoring, functioning and control: capabilities, limitations, expected accuracy, behaviour under normal use and foreseeable misuse.
Information on data: datasets used, sources, preparation methods, biases addressed.
Description of the monitoring system and metrics: traces, logs, dashboards.
Description of the QMS and the Art. 17 procedures.
FRIA where applicable (Fundamental Rights Impact Assessment, Art. 27).
Automatically generated logs that the system stores (Art. 12).
EU declaration of conformity under Art. 47, included.

The blog’s stack. It serves as a direct input for each section:

Annex IV section	Technical input from the blog
1. General description	Anatomy of the stack: 7 layers + seven deployment phases
2. Design and development	Six-stage LLMOps pipeline + continuous fine-tuning + modern alignment
3. Capabilities and limitations	Evals + LLM-as-judge
4. Data	Data versioning + RAG corpus curation
5. Monitoring	LLM tracing with OTel GenAI + prompt versioning
6. QMS	Procedures derived from ISO 42001
7. FRIA	Gap, see Art. 27 below
8. Logs	OTel tracing + retention and policies
9. Declaration of conformity	Documentary gap, see Art. 47 below

Checklist.

Annex IV dossier complete, versioned, dated, signed.
10-year retained access guaranteed (immutable / WORM storage).
Update procedure on substantial change.

Art. 12 + Art. 19 — Record-keeping (logs)

What it requires. A high-risk system must be technically capable of generating automatic logs during its operation. Those logs must allow:

Tracing how the system behaved over time.
Facilitating post-market monitoring (Art. 72).
Enabling investigation of serious incidents (Art. 73).
Supporting audits.

The provider must retain the logs for at least six months (or longer if national laws or the QMS require it). For remote biometric identification systems there are additional specific requirements.

The blog’s stack.

LLM tracing with OpenTelemetry GenAI, the canonical substrate. Every request emits a span with trace_id, gen_ai.* attributes, costs, latencies and guardrail decisions.
Prompt versioning: prompt_id + version travel as attributes.
Guardrails and safety in LLMs: gen_ai.guardrail.* attributes record every decision.
LLM Guard: spans per scanner with risk_score and action.
Data versioning: dataset_hash and model_version propagated.

Checklist.

OTel + backend (Tempo, Jaeger) running in production.
Minimum 6-month retention (24-36 months suggested for financial regulation).
WORM / immutable storage.
PII in logs redacted (via LLM Guard Vault or equivalent). Logs are not exempt from the GDPR.
Forensic query procedure with audited permissions.

Art. 13 — Transparency to deployers

What it requires. The provider gives the deployer instructions for use that are clear, complete and accessible, in comprehensible language, covering:

The provider’s identity.
Characteristics, capabilities, limitations (accuracy by category, technical specifications).
Foreseen changes to the system and its metrics.
Human oversight measures (Art. 14).
Expected computational resources and hardware.
When it applies, the expected lifetime of the system and of its maintenance.

The blog’s stack.

The OSS catalogue for LLMOps, which components and with what function.
OSS vs hyperscalers, lock-in analysis and documented dependencies.
Anatomy of an LLM request, capabilities and limitations at forensic level.

Checklist.

User manual in non-technical language for the deployer, plus a detailed technical manual.
Accuracy metrics by category with thresholds.
Change procedure with prior notification.

Art. 14 — Human oversight

What it requires. The system is designed to allow effective human oversight throughout the period of use, with interfaces and procedures that make it possible to:

Understand capabilities and limitations.
Detect malfunctions (automation bias awareness).
Decide not to use the system’s output, to override it, to reverse it.
For remote biometric identification: human verification before acting, by at least two people.

The blog’s stack.

Guardrails and safety, line 3 (Tool GR) with human-in-the-loop for destructive actions.
Evals, metrics on a human-accessible dashboard.
OTel tracing, Langfuse with auditable human sessions.

Checklist.

Oversight interface documented with use cases.
Override / abort capability with no technical restrictions.
Documented training for oversight personnel.
Metrics of the effectiveness of the oversight (override rate, false-negative rate of the oversight itself).

Art. 15 — Accuracy, robustness and cybersecurity

What it requires. High-risk systems are designed and developed to reach an appropriate level of accuracy, robustness and cybersecurity, and to perform consistently throughout their lifecycle. The relevant accuracy metrics are declared in the instructions for use. Resilience against:

Errors, faults and inconsistencies within the environment of use.
Feedback bias during operation (feedback loops).
Attacks that try to exploit the system’s vulnerabilities (data poisoning, model poisoning, model evasion, confidentiality attacks).
Technical and organisational measures to detect, respond to and resolve vulnerabilities.

The blog’s stack.

Quantisation fundamentals, accuracy vs efficiency with reported metrics.
Continuous batching + KV cache, operational robustness.
Guardrails: line 1 input + line 2 retrieval, defence against prompt injection and data poisoning via RAG.
LLM Guard: PromptGuard 2 + injection scanners, direct adversarial mitigation.
Evals: jailbreak resistance + adversarial, robustness metrics evaluated in CI.

Checklist.

Declared accuracy metrics: F1 by category, accuracy, calibration, RAG faithfulness, hallucination rate.
Robustness plan against adversarial inputs (Garak suite / Promptfoo redteam / PyRIT run periodically).
Cybersecurity plan: vulnerability management, patching of the stack (vLLM, its deps, CUDA), secrets rotation.
Analysis of potential feedback loops with drift monitoring.

Art. 17 — Quality management system (QMS)

What it requires. The provider has a written, systematic and proportionate QMS covering (without being limited to):

Regulatory compliance strategy.
Design, verification and quality control of the system.
Testing and validation procedures.
Data management.
The risk management system (Art. 9).
Post-market monitoring (Art. 72).
Incident reporting (Art. 73).
Communication with authorities, deployers and other stakeholders.
Records: documentation, log maintenance.
Resource management.
Accountability: management responsibilities.

The blog’s stack. The QMS is not code, but it leans on code.

ISO/IEC 42001 implemented (previous post) covers practically the whole content of Art. 17.
Six-stage LLMOps pipeline as the reference operating procedure.

Checklist.

QMS manual written, dated, signed, versioned.
Annual internal audit plan with criteria and records.
Management review agenda with signed minutes.
If 42001 is implemented: documented QMS-42001 mapping.

Art. 26 — Deployer obligations

What it requires. Whoever deploys the system (uses it under their own authority, not necessarily the developer) has obligations of their own:

Use the system in accordance with the instructions (Art. 13).
Assign competent, trained human oversight (Art. 14).
Ensure the input data under their control is appropriate.
Monitor operation and notify the provider on detecting problems or serious incidents.
Keep the logs under their control for at least 6 months.
Inform the affected persons when the system is used on them to make decisions (and, in an employment context, also consult their representatives).
For some Annex III cases: complete a FRIA (Art. 27).

The blog’s stack. In the case of the multi-tenant chatbot, the deployer is the client insurer. It must:

Accept the provider’s instructions (the consultancy’s) and sign the terms.
Configure human oversight on its side.
Notify the provider when it detects drift or a serious complaint.
Keep its own logs in addition to the provider’s.

Checklist (for the deployer).

Contract with the provider covering SLAs, responsibilities, exit plan.
Training programme for oversight personnel.
Incident notification procedure towards the provider.
Policy for informing affected persons (in the employment context, notification to representatives).
FRIA where applicable.

Art. 27 — Fundamental Rights Impact Assessment (FRIA)

What it requires. Applicable to deployers that are public bodies or private entities providing public services, or that use systems to assess credit score or life insurance. Before first use, the deployer carries out a FRIA documenting:

A description of the intended use.
Period and frequency of use.
Categories of affected persons.
Specific risks of harm identified.
Human oversight measures.
Mitigation measures should the risks materialise.

The FRIA is notified to the market surveillance authority. Substantial changes force an update.

The blog’s stack. The FRIA is a governance document, not a technical artefact. The blog does not cover it directly. But the technical input is:

The impact analyses from the ISO 42001 post, section A.5 are the natural starting point.
The metrics from the evals post on fairness by category and groundedness feed the risk section of the FRIA.

Checklist.

FRIA procedure documented, aligned with the AIIA in ISO 42005 (published as a complement).
FRIA carried out per system before first use.
Notification to the market surveillance authority.
Periodic review, plus on substantial change.

Art. 47 — EU declaration of conformity

What it requires. The provider draws up a written declaration of conformity per high-risk system, stating:

Identification of the system and the provider.
A statement that it complies with Arts. 8-15 + Art. 17.
Reference to the harmonised standards and common specifications applied.
Where applicable, the notified body and the certificate.
Place and date, signature and name of the authorised signatory.

It must be available to authorities for ten years. It is kept up to date.

The blog’s stack. A pure legal document. Template in Annex V.

Checklist.

Declaration of conformity signed by an authorised person before market placement.
Languages: at least the official EU language of the market where the system is placed.
Re-signing procedure on substantial change.

Art. 48 — CE marking

What it requires. A high-risk system carries the CE marking visibly, legibly and indelibly. For digital systems with no physical part, the marking is included in the documentation or the interface. If a notified body was involved, its number follows.

The blog’s stack. CE marking on a software system typically materialises in:

The product information or “About” page.
The official product documentation delivered to the deployer.
Metadata of the exposed API (custom HTTP header, OpenAPI info).

Checklist.

CE marking visible in the product interface or official documentation.
Notified body number attached where applicable.
Update procedure on substantial change.

Art. 49 — Registration in the European database

What it requires. Before placing on the market or putting into service a high-risk Annex III system (except area 2, critical infrastructure), the provider registers it in the EU database of high-risk AI systems managed by the Commission. Deployers that are public authorities also register their use. Most registration data is public (transparency towards the public).

The blog’s stack. A pure administrative procedure. No technical part beyond having the dossier ready.

Checklist.

Registration completed before first productive use.
Update on substantial change.
Portal access maintained (credentials, contact).

Art. 50 — Transparency to end users

What it requires. Applies to limited risk systems (and, complementarily, to some high-risk ones too):

Chatbots and AI assistants: the person interacting must know they are talking to an AI, unless it is obvious from the context.
Generated synthetic content (text, audio, image, video): marked as AI-generated in the output, in a machine-readable format.
Deepfakes: explicitly declare that the content is artificially generated or manipulated.
Emotion detectors or biometric categorisation: inform the affected persons.

The blog’s stack. Technical materialisation:

A UI banner in the chatbot saying “You are talking to an AI assistant”.
A disclaimer on every exportable response (PDF, email): “Generated by AI”.
Watermarking of the output (perplexity-based, model-fingerprint), optional but useful for deepfakes.
For generated audio, image or video, standard C2PA metadata.

Checklist.

Mandatory UI banner.
Disclaimer on exportable outputs.
If it generates visual content: C2PA marking or equivalent.
Procedure for use of the system on a person without their knowledge (for example, automated CV screening).

Art. 53 — Obligations of GPAI providers (in force since 2 Aug 2025)

What it requires. Providers of GPAI (general-purpose models trained with vast compute, typically foundational: Llama 4, Mistral, DeepSeek, Qwen, Gemma) must:

Maintain technical documentation of the model, accessible to the AI Office and national authorities.
Make information available to downstream providers that will integrate it.
Comply with EU copyright (Art. 4 of Directive 2019/790): an opt-out mechanism for rightsholders.
Publish a summary of the training content (Annex XI, copyright summary).

If the model has systemic risk (10^25 FLOPs threshold, or designated by the Commission), additional obligations apply (Art. 55): model evaluation, adversarial testing, incident reporting, cybersecurity.

The blog’s stack. For an organisation that uses GPAI models (Llama 4, Mistral) and does not train them from scratch:

It is not a GPAI provider; it is a downstream provider integrating GPAI into its system.
It must hold the GPAI technical documentation (Llama paper, Mistral docs) and reference it in its own Annex IV documentation.
Analysis of the specific GPAI licence (Llama Community License, Apache 2.0, and so on).
The post on modern alignment and continuous fine-tuning describe how a LoRA adapter on top of the GPAI does not turn the downstream party into a GPAI provider, as long as it does not train a new model from scratch.

Checklist (downstream provider).

Inventory of GPAI models used, with version, source, licence, referenced documentation.
Mapping of responsibilities between the upstream GPAI provider and us as downstream.

Art. 72 — Post-market monitoring

What it requires. A documented post-market monitoring system proportionate to the risk, collecting data on performance throughout the lifecycle, including interaction with other AI systems. It allows the provider to:

Assess continuous compliance with Arts. 8-15.
Adopt the necessary corrective measures.
Detect trends in real use (drift, abuse).

The monitoring plan is part of Annex IV and is maintained throughout the system’s life. The Commission publishes a template in 2026.

The blog’s stack.

OTel tracing + Langfuse, the base.
Continuous evals, operational metrics as online gates.
LLM-as-judge, judges over production sampling to detect degradation.
Incident-driven retrain, the closed improvement loop.

Checklist.

Post-market monitoring plan documented per system.
Operational metrics defined with thresholds and periodic review.
Corrective action procedure on alert.
Integration with Art. 73 (when an alert equals a serious incident).

Art. 73 — Serious incident reporting

What it requires. The definition of a serious incident (Art. 3(49)):

Death or serious harm to health.
Serious disruption of critical infrastructure.
Infringement of legal obligations intended to protect fundamental rights.
Serious damage to property or the environment.

The provider reports to the market surveillance authority of the Member State where the incident occurred. Deadlines:

Type of incident	Maximum deadline from becoming aware
General	15 days
Death	10 days
Critical infra affected, or widespread infringement	2 days

The first report may be preliminary; the complete one follows later. Internal investigation is mandatory, as is cooperation with the authorities. Corrective action is proportionate.

The blog’s stack. The technical flow:

OTel tracing provides the traceability for forensic investigation.
Guardrails and safety emits the canonical incident_event (category, severity, trace_id).
Incident-driven retrain materialises the corrective action.

Checklist.

Incident reporting procedure documented with deadlines and templates.
Designated person (typically DPO + AI Risk Owner) with responsibility and training.
Annual dry-run of the procedure (drill).
Technical integration between the guardrails / tracing layer and the notification channel.

The assembled Annex IV dossier: SVG of the complete file

The diagram shows the blog’s editorial asymmetry: seven of the nine Annex IV sections are covered directly by the technical stack. Only the FRIA (section 7) and the signed declaration of conformity (section 9, not shown in the SVG because it is a one-page document) are documentary gaps needing express administrative work. The main compliance effort is assembling and signing, not building from scratch.

Applied case: the multi-tenant chatbot assessed against the AI Act

We take the system from the forensic post, a multi-tenant customer support chatbot for insurers, and walk through it as the provider of the system.

Art. 6 classification: the chatbot helps customers understand products, check status and open tickets. It does not automate contractual decisions (it does not approve claims, it does not calculate premiums). Does it fall into Annex III area 8 (essential private services)? That depends on the use. If the insurer uses it only for informational support, it is limited risk (Art. 50 applies). If it uses it to assess claim declarations, it is high risk. Documenting the analysis is mandatory.

We assume high risk for the complete walkthrough:

Art. 5 prohibitions: written statement of non-applicability. ✓
Art. 9 risk management: per-system document with identified risks (hallucination, dialect bias, PII leakage, jailbreak), applied mitigations (RAG over a curated corpus, guardrails with 4 lines, LLM Guard Vault, continuous evals) and signed accepted residuals. ✓ (input: pipeline + guardrails + evals).
Art. 10 data governance: governance of the four datasets (training adapter, insurer RAG corpus, golden eval, enriched retrain) with biases analysed, PII anonymised, lineage in place. ✓ (input: data-versioning + rag-corpus-curation + LLM Guard).
Art. 11 + Annex IV: dossier with 9 sections drafted, signed, accessible for 10 years in a WORM bucket. ✓ (7 sections from the blog, 2 new).
Art. 12 + Art. 19 logs: OTel + Tempo + Langfuse with 24-month retention, PII redacted by Vault. ✓
Art. 13 transparency to deployers: user manual for the insurer plus a technical manual. ✓ (input: OSS catalogue + request anatomy).
Art. 14 human oversight: Langfuse + Grafana dashboard, human escalation protocol for critical cases, training for the insurer’s staff. ✓
Art. 15 accuracy, robustness, cybersecurity: F1 by category declared, Promptfoo redteam adversarial suite run monthly, stack cybersecurity plan (vLLM patching, secrets rotation). ✓
Art. 17 QMS: ISO 42001 implemented and certified. ✓
Art. 26 deployer: the contract with the insurer includes the deployer’s obligations. ✓
Art. 27 FRIA: the insurer carries out a FRIA before first use (it is a private essential-service entity). ✓ (deployer’s responsibility, the provider assists).
Art. 47 declaration of conformity: signed by the consultancy’s CTO before market placement. ✓
Art. 48 CE marking: visible in the chatbot interface and in the official documentation. ✓
Art. 49 EU DB registration: completed before first productive use. ✓
Art. 50 transparency to users: UI banner “You are talking to an AI assistant”, disclaimer on exportable responses. ✓
Art. 53 GPAI: Llama 4 documentation (the base model) referenced, plus EU copyright analysis and responsibility mapping. ✓
Art. 72 post-market monitoring: documented plan, OTel + continuous evals + retrain already running. ✓
Art. 73 incident reporting: procedure, designated owner, annual dry-run. ✓

Result: certifiable and deployable on the EU market on 2 August 2026. The key gaps (FRIA, CE marking, EU DB registration, declaration of conformity) are documentary work over technical artefacts that already exist, not new technical projects.

Penalties (Art. 99)

The penalty table is proportional to worldwide turnover or to an absolute cap, whichever is higher:

Violation	Penalty cap
Art. 5 (prohibited practices)	Up to 35 M€ or 7% of annual worldwide turnover
Other obligations (Arts. 8-22, 26-50, 72-73, and so on)	Up to 15 M€ or 3% of annual worldwide turnover
Incorrect or misleading information to authorities	Up to 7.5 M€ or 1% of annual worldwide turnover

For SMEs and startups the caps are the lower of the two figures (not the higher). The proportionality mitigation exists but requires formal demonstration.

In addition, national authorities can order:

Immediate suspension of the system on the market.
A mandatory recall.
Mandatory public communication of the penalty.

The five frequent compliance traps

Trap 1 — Assuming the GPAI model used already covers the obligations. The downstream provider remains responsible for the integrated system: neither Meta nor Mistral nor DeepSeek takes on the obligations of whoever builds on their models. The trap surfaces when the authority asks for the dossier and the team points at the Llama model card as if that were enough.

Trap 2 — Confusing ISO 42001 with EU AI Act conformity. Holding a 42001 certificate does not imply conformity: certification is not a FRIA, not CE marking, not EU database registration, not a signed declaration of conformity. Standardisation is progressing, but until ISO 42001 is published as a harmonised standard (it was not at the end of 2025), there is no presumption of conformity. The table of legal obligations runs on its own circuit.

Trap 3 — Forgetting incident reporting within the legal deadline. Fifteen days sounds long until an incident lands in the middle of the summer holidays. Two days for critical infrastructure is not negotiable. Without a documented procedure and an annual drill, the deadline is missed in silence. A penalty for misleading information (1% of worldwide turnover) follows if the incomplete reporting is discovered.

Trap 4 — Underestimating the deployer. The AI Act assigns obligations to both the provider and the deployer. A company that uses a hosted LLM integrated into its service (without developing it) is still a deployer with obligations of its own (Art. 26): human oversight, FRIA where applicable, notification to affected persons. The trap surfaces when the deployer assumes that “responsibility lies with the model provider”. It does not, not entirely.

Trap 5 — Leaving the FRIA until last. The FRIA (Art. 27) requires a fundamental rights impact analysis with qualitative dimensions (discrimination, privacy, social rights). It is not an afternoon’s document. It is carried out before first use, not after a problem is detected. Public deployers and private essential-service providers who leave it for “whenever they ask” take 4-8 weeks to produce a credible one, time they typically do not have when the inspection arrives.

What we have not covered (upcoming posts)

Concrete templates for each document: EU declaration of conformity (Annex V), full Annex IV dossier section by section, FRIA, post-market monitoring plan, initial serious incident report. Material for a post along the lines of “The EU AI Act compliance folder in 12 templates”.
Voluntary codes of practice published by the Commission under Art. 56, useful for limited-risk systems that want to demonstrate good behaviour without a legal obligation.
Comparative analysis of notified bodies for systems requiring third-party conformity assessment (mainly Annex I).
How compliance changes for LLM agents, systems with graduated autonomy, tool calling and the capacity to act. SC 42 and the AI Office are working on specific guidance.
The complete map of national authorities designated under Art. 70, plus the sectoral authorities that retain jurisdiction (AEPD for privacy, CNMV for financial services, Banco de España for banking, AESA for aviation).

References

Regulation (EU) 2024/1689 (EU AI Act) — Texto consolidado en EUR-Lex: https://eur-lex.europa.eu/eli/reg/2024/1689. Diario Oficial L 1689/12.7.2024.
EU AI Act Explorer (AI Act Service Desk, Comisión Europea): https://ai-act-service-desk.ec.europa.eu/en/ai-act-explorer.
AI Act Text portal (artificialintelligenceact.eu): artículos individuales con anotaciones.
Anexo IV — Technical documentation: estructura de los nueve apartados obligatorios.
Anexo V — EU declaration of conformity: plantilla obligatoria.
Anexo III — High-risk AI systems: las ocho áreas que clasifican un sistema como alto riesgo.
Anexo XI — Copyright training summary template: para GPAI providers.
NIST AI RMF 1.0 (2023) — https://www.nist.gov/itl/ai-risk-management-framework.
ISO/IEC 42001:2023 — sistema de gestión, complemento facilitador.
ISO/IEC 42005 — Impact assessment AI (publicada 2025 como guía técnica para FRIA).
Draft Commission guidance on serious incident reporting (2025) — borrador en consulta para Art. 73.

Nis2 on lo0 — Tech Blog

Hardening and secrets in the sovereign LLM stack: defence in depth

TL;DR

The analogy: the sovereign office

Layer 1 — Secrets: GitOps’s chicken-and-egg problem

Family A — sealed-secrets (asymmetric encryption, controller in the cluster)

Family B — SOPS + age (file encryption, simple)

Family C — External Secrets Operator + Vault (dynamic secrets, native rotation)

How to choose, and rotation

Layer 2 — Network: default-deny, egress and the arithmetic of the surface

The arithmetic of communication pairs

Standard L3/L4, L7 and egress with Cilium

Internal mTLS

Layer 3 — Pod security: restricted, and nothing privileged

Layer 4 — Image supply chain

Layer 5 — AuthN/Z: virtual keys, OIDC and minimal RBAC

Layer 6 — Runtime: the cameras with Tetragon

Layer 7 — Data at rest

Exposure table: service × port × external egress?

Blast radius: a leaked secret, with and without defence

The ENS / NIS2 angle

Applied to the generic 4×H100 cluster

What hardening does NOT solve

See also

Incident response runbooks for LLM inference: every alert to a concrete action with Kafka and Keep

TL;DR

You are here: OBSERVE → DEPLOY (incident response closes the loop)

The analogy: a nuclear reactor control room

The incident pipeline architecture

The six critical alerts and their runbooks

RB-01 · GpuHbmNearOom — HBM > 92 % sustained

RB-02 · GpuThermalOrPowerThrottle — bit ≠ 0 and not Idle in CLOCK_THROTTLE_REASONS

RB-03 · GpuXidErrorDetected — increase(DCGM_FI_DEV_XID_ERRORS[5m]) > 0

RB-04 · GpuEccDoubleBit — DCGM_FI_DEV_ECC_DBE_VOL_TOTAL > 0

RB-05 · VllmKvCachePoolNearFull — gpu_cache_usage_perc > 95 % sustained 3 min

RB-06 · VllmTtftP95OutOfSlo — TTFT P95 > 1.5 s for 5 min

Keep YAML workflows — three complete examples

Workflow 1 — xid-detected.yaml

Workflow 2 — ecc-dbe.yaml — immediate paging

Workflow 3 — canary-rollback.yaml — TTFT P95 out of SLO

The canonical Kafka event schema

Formal fit into incident management

ISO/IEC 27035 — information security incident management

ENS (Esquema Nacional de Seguridad) — op.exp controls

NIS2 — notification to the competent authority

EU AI Act — art. 73 (serious incident reporting)

ISO/IEC 42001 — AIMS clause 10 continual improvement

Four anti-patterns

Applied to typical on-premise hardware

What we have not covered (upcoming posts)

See also

References

ISO/IEC 42001: the operations manual for the AI system — how the AIMS fits over the on-premise LLM platform described in this blog

TL;DR

The analogy: the aircraft operations manual

ISO/IEC 42001 in 15 seconds

Distinction from neighbouring frameworks

The seven clauses (Annex SL): the mandatory table of contents

Clause 4 — Context of the organisation

Clause 5 — Leadership

Clause 6 — Planning

Clause 7 — Support

Clause 8 — Operation

Clause 9 — Performance evaluation

Clause 10 — Improvement

The 38 Annex A controls: the AI-specific catalogue

Cross-mapping: 38 controls ↔ blog posts

A.2 — AI policies (2 controls): PARTIAL

A.3 — Internal organisation (3 controls): GAP

A.4 — Resources (6 controls): COVERED

A.5 — Impact assessment (5 controls): PARTIAL

A.6 — AI system lifecycle (4 controls): COVERED

A.7 — Data for AI systems (5 controls): COVERED

A.8 — Information for interested parties (4 controls): COVERED

A.9 — Use of AI systems (3 controls): COVERED

A.10 — Third parties and customer relationships (4 controls): COVERED

The roles defined by the standard

Impact levels and proportionality

The seven minimum AIMS documents

Applied case: the blog’s multi-tenant chatbot → 42001 checklist

RB-01 · `GpuHbmNearOom` — HBM > 92 % sustained

RB-02 · `GpuThermalOrPowerThrottle` — bit ≠ 0 and not Idle in CLOCK_THROTTLE_REASONS

RB-03 · `GpuXidErrorDetected` — `increase(DCGM_FI_DEV_XID_ERRORS[5m]) > 0`

RB-04 · `GpuEccDoubleBit` — `DCGM_FI_DEV_ECC_DBE_VOL_TOTAL > 0`

RB-05 · `VllmKvCachePoolNearFull` — `gpu_cache_usage_perc > 95 %` sustained 3 min

RB-06 · `VllmTtftP95OutOfSlo` — TTFT P95 > 1.5 s for 5 min

Workflow 1 — `xid-detected.yaml`

Workflow 2 — `ecc-dbe.yaml` — immediate paging

Workflow 3 — `canary-rollback.yaml` — TTFT P95 out of SLO