Audit on lo0 — Tech Blog

Virtual keys, budgets and limits in LiteLLM: the layer that decides who gets the GPU, and the four things the documentation gets wrong

Tue, 08 Sep 2026 07:30:00 +0200

Third article in the operational track of the control layer. The pair with Langfuse covered observability, and day 2 of the proxy covered availability. Here it is governance: who can call, which model, how much and on what budget. The economic model behind those figures is in FinOps and multi-tenancy with LiteLLM.

TL;DR

On a platform running on your own GPUs there is no provider invoice to put consumption in order. Capacity is fixed, known and shared, so someone has to impose the allocation, and that someone is the gateway.

The hierarchy has six levels: organisation, team, team member, internal user, key and end customer, plus the project and tag scopes. Each level’s budgets are evaluated independently and all of them have to pass. They neither add up nor cancel each other out, with one exception.

The exception is in the code and it is the most important governance trap in the system. A key that belongs to a team suppresses the check of its owner’s personal budget, unless apply_user_budget_to_team_keys is enabled. A user with a hundred-euro cap who works with their team’s key spends whatever the team allows.

Rotating a key preserves its spend. /key/regenerate does an UPDATE on the same row changing the hash, so spend, max_budget and the budget window remain intact. It accepts a grace period, stored in a table of its own, and if the period’s format is wrong it is ignored without blocking anything.

The v3 limiter is already the default in 1.100.0, and you go back to the previous one with an environment variable. Its window is sliding, sixty seconds, anchored to the first request, and the counters are incremented with Lua scripts inside Redis using Redis’s own timestamp to close the race between replicas.

And four points where the official documentation does not match the code, verified against tag v1.100.0: an exhausted budget returns 429, not 400; the audited actions are six, not three; /key/info and /key/list are GET, not POST; and the v3 limiter carries a docstring saying it is not production-ready while it is the one loaded by default.

You are here: the gateway as an administrative boundary

Without this layer, an in-house inference platform has a single mode of operation: whoever knows the URL consumes. It works while there is one team. It stops working the day there are three, or the day someone connects an agent that fires twenty calls per interaction and wrecks everyone else’s time to first token.

The two questions this layer answers are different and have to be kept apart. The capacity one is instantaneous: how many requests and how many tokens per minute each consumer can push, so the fleet does not saturate and latency stays within the service agreement. The cost one is cumulative: how much each consumer has spent this month against its budget. They are configured in the same place, applied at different moments and fail in different ways.

The analogy: the building’s key ring

An office building hands out keys at three levels. The master opens everything and the building manager has it. The floor keys open a whole floor and each tenant company has one. And the office keys open one office.

Three properties of that system are the ones that matter here. A key is replaced without changing the lock of the whole building, and the tenant keeps their parking space and their electricity meter: that is rotation preserving the spend. A lost key is cancelled from the central panel without collecting anything physical: that is blocking from the interface. And the record of who took out each key and when lives in a separate book, which is what you show an inspector: that is the audit table.

The analogy breaks at one point, and that point is where this system’s governance flaw lives. In the building, having a floor key does not cancel the office’s consumption limit. In LiteLLM, it does.

The hierarchy, and the line that breaks it

The levels and their management endpoints:

Level	Endpoints	Budget
Organisation	`/organization/new`, `/organization/member_add`, …	`max_budget`
Team	`/team/new`, `/team/member_add`, `/team/block`, …	`max_budget`
Team member	`/team/member_add` with `max_budget_in_team`	Its own row in the budgets table
Internal user	`/user/new`, `/user/update`, …	`max_budget`
Key	`/key/generate`, `/key/update`, …	`max_budget`, `soft_budget`
End customer	`/customer/new`, and the parallel `/end_user/*` set	`max_budget`

The default behaviour is the right one for a multi-tenant platform: each scope carries its own counter and the check of each one is independent of the rest. A request passes if all of them pass.

And then there is this condition, which lives in the authorisation checks:

is_team_key = team_object is not None and team_object.team_id is not None
if is_team_key and general_settings.get("apply_user_budget_to_team_keys") is not True:
 return

Translated: when the key belongs to a team, the check of the budget of the user who owns that key returns without evaluating anything. The personal cap exists in the database, is visible in the interface, and is not applied.

For a deployment where personal budgets are decorative, it makes no difference. For one where the per-person cap is the control that was shown to someone as a guarantee that a user cannot overrun the platform, it is a hole. The line that closes it is one:

general_settings:
 apply_user_budget_to_team_keys: true

Virtual keys: the complete life cycle

The endpoints all exist and some of them are not where you expect:

Route	Method	What for
`/key/generate`	POST	Create
`/key/update`	POST	Modify
`/key/info`	GET	Query one
`/key/list`	GET	List
`/key/delete`	POST	Delete
`/key/regenerate`	POST	Rotate
`/key/block`, `/key/unblock`	POST	Suspend and reactivate
`/key/health`	POST	Check that key’s callbacks

The two marked in bold are GET, and several examples in the documentation suggest otherwise. /key/health is not what its name suggests either: it does not validate the key or its budget, it checks the logging callbacks associated with it.

There are many creation parameters, and these are the ones that govern:

curl -X POST 'https://gateway.interno/key/generate' \
 -H 'Authorization: Bearer sk-...' \
 -d '{
 "key_alias": "equipo-datos-notebooks",
 "team_id": "t-datos",
 "models": ["llama-70b", "qwen-30b"],
 "max_budget": 250,
 "budget_duration": "30d",
 "tpm_limit": 400000,
 "rpm_limit": 600,
 "max_parallel_requests": 8,
 "duration": "90d",
 "tags": ["produccion", "notebooks"]
 }'

Alongside those there are controls that are rarely used and solve specific problems: model_max_budget to set a per-model cap within the same key, model_tpm_limit and model_rpm_limit for the same thing with capacity, enforced_params to require certain keys to always arrive with given fields, allowed_routes so an application key cannot touch the management endpoints, and blocked to be born suspended.

How they are stored

A key’s hash is a single-pass hexadecimal SHA-256, with no salt and no derivation function:

hashed_token = hashlib.sha256(token.encode()).hexdigest()

The row lives in LiteLLM_VerificationToken, with the hash as primary key, and it contains the accumulated spend, the limits, the expiry, the allowed models and the rest of the configuration.

Two consequences to be clear about before an audit. The first is that this hash is not designed to resist a dictionary attack against the database, and the code itself acknowledges it in a comment about encryption key derivation, where it admits that a single-pass unsalted SHA-256 is not a derivation function and that moving to HKDF would be more defensible in an audit. The real mitigation is that virtual keys are generated with enough entropy, not the strength of the hash. The second is that LITELLM_SALT_KEY plays no part here: that variable encrypts the stored provider credentials, not the virtual keys.

There is a trap around that variable that takes down a whole deployment. If it is not defined, it falls back to the master key. Rotating LITELLM_MASTER_KEY without having first set your own LITELLM_SALT_KEY leaves every stored provider credential unreadable, and the failure does not block: it logs an error and returns null, so the symptom shows up as models that stop authenticating with no explanation. Setting LITELLM_SALT_KEY on day one, and never touching it, is one of the things that saves the most grief.

Rotation

/key/regenerate does not create a new key and delete the old one. It updates the same row changing the hash, so the accumulated spend, the maximum budget and the reset window are preserved. For a monthly chargeback this is exactly what you want: rotating mid-month does not zero the counter.

It also accepts a grace period, so the old key remains valid while consumers update:

curl -X POST 'https://gateway.interno/key/sk-vieja/regenerate' \
 -H 'Authorization: Bearer sk-...' \
 -d '{"grace_period": "24h"}'

The old hash is stored in a separate table with its revocation date. Without that parameter, revocation is immediate, and the default value can be set with LITELLM_KEY_ROTATION_GRACE_PERIOD.

A detail with teeth: if the period’s format is wrong, the system logs a warning and continues with no grace period. The rotation completes, the old key dies on the spot, and all that is left is a log line. In an automated rotation procedure, that combination of “carry on” and “warn in the log” is what produces the outage at three in the morning.

What exactly happens when a budget runs out

The code raises BudgetExceededError with status 429. The documentation says 400 in one place, and there the code wins. Its being a 429 has a practical implication: OpenAI clients retry automatically on a 429, so an exhausted budget turns into a retry storm against a gateway that is going to keep rejecting. Internal clients should distinguish the reason, which travels in the message.

The messages are templates and state the scope, the spend and the cap:

Budget has been exceeded! Key=<clave> Current cost: <gasto>, Max budget: <tope>
Budget has been exceeded! Team=<equipo> Current cost: <gasto>, Max budget: <tope>
ExceededBudget: User=<usuario> over budget. Spend=<gasto>, Budget=<tope>
ExceededBudget: End User=<cliente> over budget. Spend=<gasto>, Budget=<tope>
Budget has been exceeded! Organization=<org> ...
Budget has been exceeded! Tag=<etiqueta> ...

Two clarifications about the comparison. It is spend >= cap, with the equals sign included, so landing exactly on the cap already blocks. And the soft budget, soft_budget, blocks nothing: it only writes a log line when it is crossed. It serves to warn, not to contain, and building an alert on top of it is the right way to use it.

The rest of the rejections, with their codes:

Situation	Code	Behaviour
Budget exhausted	429	Message with scope, spend and cap
Expired key	401	It also deletes the cache entry, so the expiry takes effect across the whole fleet without waiting for the TTL
Model not allowed	403	Different error type depending on the scope: key, team, user, organisation or project
Blocked key	Generic exception	`Key is blocked. Update via /key/unblock if you're an admin.`

The last row is an inconsistency in the project worth knowing about: blocking raises an exception with no explicit type or code, while expiry and model access do carry them. Any alert that discriminates by error type will not see blocks the way it sees the rest.

A useful note about allowed models: the check accepts both the alias and the underlying model, so a key authorised for llama-70b can call it by its alias or by its real name with no additional configuration.

Capacity limits: the v3 limiter

This is the most significant silent change of the year in this layer. In 1.100.0, the limiter loaded by default is the third version. Going back to the previous one has to be asked for explicitly:

LEGACY_MULTI_INSTANCE_RATE_LIMITING=true

There is no flag to turn it on, only to turn it off. And it carries a contradiction worth knowing before trusting it: the file itself still opens with a comment saying it is under development and not production-ready, while it is the one running. The design is avowedly inspired by Envoy’s.

The window is sliding and lasts sixty seconds, configurable with LITELLM_RATE_LIMIT_WINDOW_SIZE. It is not an exact rolling log nor a clock-aligned window: it anchors to the first request and resets once the window size has elapsed from that moment. For one-minute limits, the difference from a calendar window shows up at the edges and rarely matters.

What does matter is Redis. The counters are incremented with Lua scripts executed inside Redis, using Redis’s own timestamp and not the pod’s, precisely so window resets are deterministic across replicas and to close the race between checking and counting. Without Redis, each pod counts on its own and the effective limit is multiplied by the number of pods. A limit of 600 requests per minute with five replicas is a limit of 3,000.

There is an inherent imprecision in the token limits that Redis does not solve. The number of output tokens is not known until the response finishes, so the limiter counts with an estimate before the call and reconciles afterwards. The possible excess is, per in-flight request, the difference between the estimate and the real figure. With high concurrency and long responses, that allows the cap to be exceeded within a window. To protect the fleet, the concurrent request limit (max_parallel_requests) is a more direct instrument than tokens per minute.

What the client sees

When limited, a 429 with the message giving the scope, the current limit, what is left and when it resets. And three headers: retry-after with the window size, that is 60 by default, plus rate_limit_type and reset_at.

On the success path, the informational headers use a nested format of their own:

x-ratelimit-api_key-remaining-requests
x-ratelimit-api_key-limit-tokens
x-ratelimit-team-remaining-requests

They are not the standard names x-ratelimit-limit-requests that OpenAI and compatibles emit, so a client expecting the usual format will find nothing. There is also an open issue about these headers being lost on streaming responses, which is most of an assistant’s traffic.

Identity: beyond the static key

Handing out keys by hand works with three teams and stops working with thirty. The two routes for connecting the gateway to corporate identity:

JWT. It is enabled with enable_jwt_auth and configured with the litellm_jwtauth block, which maps token fields to LiteLLM entities: the team comes from team_id_jwt_field, the user from user_id_jwt_field, the organisation from org_id_jwt_field. It supports dot notation for nested claims, automatic creation of users and teams with user_id_upsert and team_id_upsert, role-based access control with enforce_rbac, per-team model restriction with enforce_team_based_model_access, and several issuers at once. The issuer’s public key is cached with a TTL of 600 seconds by default.

general_settings:
 enable_jwt_auth: true
 litellm_jwtauth:
 team_id_jwt_field: "groups"
 user_id_jwt_field: "sub"
 user_email_jwt_field: "email"
 enforce_rbac: true
 enforce_team_based_model_access: true
 team_id_upsert: true

OIDC for the admin interface. With Keycloak you use the generic provider, and the variables are PROXY_BASE_URL, GENERIC_CLIENT_ID, GENERIC_CLIENT_SECRET, GENERIC_AUTHORIZATION_ENDPOINT, GENERIC_TOKEN_ENDPOINT, GENERIC_USERINFO_ENDPOINT, plus a set of attributes to map the identifier, the email, the name and the role, and GENERIC_ROLE_MAPPINGS_* to translate Keycloak groups into LiteLLM roles. Login is rate-limited and sessions have a TTL of their own.

For automatic provisioning there is SCIM in the enterprise version, with base /scim/v2, and deprovisioning a user blocks their keys and takes them out of the authentication cache. That part is not in the open tree, so it cannot be verified against the code.

This fits with the identity work already covered in hardening and secrets of the sovereign stack and in the article on MCP authentication with Keycloak, which uses the same issuer.

Audit: what gets recorded

The table is LiteLLM_AuditLog and its columns are what you would expect from a change log:

id, updated_at, changed_by, changed_by_api_key,
action, table_name, object_id, before_value, updated_values

It stores the previous value and the new one, and attributes the change to an actor and to the key it was made with.

Three clarifications the documentation does not capture well.

The actions are six, in the past tense: created, updated, deleted, blocked, unblocked and rotated. The documentation mentions three. That blocking, unblocking and rotation are actions in their own right is exactly what is needed to reconstruct a credential’s life cycle for an auditor.

The scope is wider than advertised. Besides teams, users, keys and models, changes to the proxy configuration and to the SSO configuration are audited. A change of identity issuer is recorded.

The default depends on the licence. The resolution order is litellm_settings.store_audit_logs, then the variable LITELLM_STORE_AUDIT_LOGS, and if neither is set, it stays enabled in enterprise and disabled in the open version. An OSS deployment that assumes there is an audit trail does not have one:

litellm_settings:
 store_audit_logs: true

Two more pieces for a serious chain of evidence. The actor attribution can be passed with the LiteLLM-Changed-By header, and its use is restricted by the key’s or the team’s configuration, so any caller cannot forge it. And the events can be shipped out with audit_log_callbacks and its S3 parameters, which is what allows depositing them in an immutable store with object lock.

That last part is what turns the record into evidence. The Postgres table is mutable by anyone with database access, and as of September 2026 it has no indexes, so querying it by date range over a long history will not be comfortable either.

What this demonstrates to an auditor, and what it does not

This is the part that decides whether the setup is good enough for a regulated customer. With everything above configured, the platform sustains without difficulty:

Access control via corporate identity, with onboarding and offboarding in the identity provider and not in the gateway, and with the role determining which models are reachable. Segregation between tenants, with independent counters per organisation, team and person. Traceability of administrative changes, with actor, time, previous value and new value, exportable to an immutable store. And demonstrable consumption limits, with the evidence of the rejection in the spend log.

What it does not cover, and has to be solved elsewhere:

The record of the interactions is not this. The audit table records configuration changes, not model calls. The calls are in the spend log and in the traces, and traces are best-effort, which is the discussion in the first article in the series.

The integrity of the record is not something Postgres provides. Without export to immutable storage, the evidence is only as strong as the database permissions.

The content of the prompts is a separate problem, with its own personal-data treatment and its own decision about whether it is stored, masked or not logged at all.

And a warning about the control that tends to be shown first: while apply_user_budget_to_team_keys is not enabled, the per-person cap is not being applied to team keys. Showing that field in the interface as proof of a control that is not executed is the kind of finding a competent auditor discovers. The complete control mapping is in ENS, ISO 42001 and the EU AI Act.

Checklist

apply_user_budget_to_team_keys: true if personal budgets have to be enforced.
LITELLM_SALT_KEY set on day one, different from the master key and never rotated lightly.
store_audit_logs: true explicitly, without trusting the default.
audit_log_callbacks towards an immutable store if an audit has to be sustained.
Redis mandatory if there is more than one replica, or the limits get multiplied by the number of pods.
max_parallel_requests per key, on top of the per-minute limits, to protect the fleet from a runaway agent.
allowed_routes on application keys, so they cannot reach the management endpoints.
soft_budget with an alert on top, understanding that it does not block.
Rotation with a verified grace_period, because an invalid format revokes on the spot.
JWT against Keycloak as soon as there are more than a handful of teams, with enforce_rbac and enforce_team_based_model_access.

Traps and things that are not what they look like

A user’s cap is not enforced. They are using a team key and apply_user_budget_to_team_keys is missing.

Clients retry in a loop against an exhausted budget. The code is 429 and the OpenAI libraries retry 429s automatically.

The soft budget has stopped nothing. It does not block, it only logs.

The per-minute limits are enforced at double or triple. There is no Redis, and each replica keeps its own count.

The client does not see the limit headers. The names are nested and proprietary, and in streaming there is an open issue where they get lost.

There is no audit log. It is an OSS deployment and the default there is disabled.

The models stop authenticating after rotating the master key. There was no fixed LITELLM_SALT_KEY, and the encrypted provider credentials can no longer be decrypted.

The rotation cut the service on the spot. The grace_period carried a format the system did not understand and it carried on without it.

Blocking a key does not fire the alert. It is raised as a generic exception, without the type that expiry and model access do carry.

Closing

The keys and budgets layer is the one that turns a shared URL into a platform with tenants. It is well solved in LiteLLM and has more depth than the documentation suggests, with per-model budgets inside a key, multiple budget windows, per-tag limits and rotation that preserves the history.

Out of the whole article, two things take the attention. The apply_user_budget_to_team_keys line, because it is the difference between having a control and believing you have one. And setting LITELLM_SALT_KEY, because its default turns a routine rotation of the master key into an outage that is hard to diagnose. Both are one line of configuration and both are discovered late.

Sources

LiteLLM, Virtual Keys (parámetros de creación, ciclo de vida, rotación): https://docs.litellm.ai/docs/proxy/virtual_keys.
LiteLLM, Budgets, Rate Limits (jerarquía, presupuestos por nivel, clientes finales): https://docs.litellm.ai/docs/proxy/users.
LiteLLM, JWT Token Auth (bloque litellm_jwtauth y todos sus campos): https://docs.litellm.ai/docs/proxy/token_auth.
LiteLLM, Audit Logs y administración múltiple (la cabecera LiteLLM-Changed-By): https://docs.litellm.ai/docs/proxy/multiple_admins.
LiteLLM, SCIM (aprovisionamiento automático, versión enterprise): https://docs.litellm.ai/docs/tutorials/scim_litellm.
LiteLLM, litellm/proxy/auth/auth_checks.py en la etiqueta v1.100.0 (la condición de apply_user_budget_to_team_keys y los mensajes de presupuesto agotado): https://github.com/BerriAI/litellm/blob/v1.100.0/litellm/proxy/auth/auth_checks.py.
LiteLLM, litellm/proxy/management_endpoints/key_management_endpoints.py en v1.100.0 (rutas, rotación sobre la misma fila, periodo de gracia): https://github.com/BerriAI/litellm/blob/v1.100.0/litellm/proxy/management_endpoints/key_management_endpoints.py.
LiteLLM, litellm/proxy/hooks/parallel_request_limiter_v3.py y litellm/proxy/hooks/__init__.py en v1.100.0 (el limitador v3 como valor por defecto, la ventana, los scripts Lua): https://github.com/BerriAI/litellm/blob/v1.100.0/litellm/proxy/hooks/parallel_request_limiter_v3.py.
LiteLLM, litellm/exceptions.py en v1.100.0 (BudgetExceededError con código 429): https://github.com/BerriAI/litellm/blob/v1.100.0/litellm/exceptions.py.
LiteLLM, litellm/proxy/common_utils/encrypt_decrypt_utils.py en v1.100.0 (LITELLM_SALT_KEY cayendo en la clave maestra, y el comentario sobre la derivación de claves): https://github.com/BerriAI/litellm/blob/v1.100.0/litellm/proxy/common_utils/encrypt_decrypt_utils.py.
LiteLLM, schema.prisma en v1.100.0 (LiteLLM_VerificationToken, LiteLLM_AuditLog, la tabla de claves deprecadas): https://github.com/BerriAI/litellm/blob/v1.100.0/schema.prisma.
LiteLLM, issue #27748 — cabeceras x-ratelimit-* perdidas en respuestas en streaming con el limitador v3: https://github.com/BerriAI/litellm/issues/27748.

ISO/IEC 42001: the operations manual for the AI system — how the AIMS fits over the on-premise LLM platform described in this blog

Mon, 01 Jun 2026 06:00:00 +0200

This post closes an asymmetry the blog had been accumulating: we have described in detail the technical platform (seven stack layers, seven deployment phases, five maturity levels), the operational pipeline (six LLMOps stages), the data pieces (corpus curation, versioning), the eval / safety pieces (evals, guardrails, LLM Guard) and the observe pieces (OTel GenAI tracing). What had not appeared is the governance layer a regulated client asks for on top of all that. ISO/IEC 42001 is that layer.

TL;DR

ISO/IEC 42001:2023 is the first certifiable international standard that defines how an AI system is managed. It is not a technical standard (it does not say “use this inference engine” or “this safety threshold”): it is a management standard, a cousin of ISO 27001 and ISO 9001. From both it inherits the Annex SL structure, seven mandatory clauses running through context, leadership, planning, support, operation, performance evaluation and improvement, and it adds an Annex A with 38 AI-specific controls in 9 sections: policies, internal organisation, resources, impact assessment, lifecycle, data, information to interested parties, use, third parties. The thesis of this post is that the technical architecture described in this blog directly covers between 60% and 80% of the A controls with no extra work. The LLMOps pipeline materialises A.6, versioning and curation materialise A.7, guardrails and evals materialise A.9, OTel tracing materialises A.8. The rest is governance discipline that does not appear in the code (a written AI policy, impact assessments per system, a stakeholder register, role decisions between provider/producer/customer, mandatory documentation), and that is precisely what separates a real certification from performative compliance. The post maps the correspondence control by control, cross-references it with the EU AI Act (seven articles directly aligned with 42001: 9, 10, 11, 12, 13, 14, 17), with NIS2 (asset register, incident notification, supply chain) and with ENS (Esquema Nacional de Seguridad, Spain’s national security framework, RD 311/2022, Basic/Medium/High categories), lists the seven mandatory minimum documents an auditor asks for, presents the blog’s multi-tenant chatbot as a live 42001 checklist, and closes with the five usual traps (confusing 42001 with EU AI Act compliance, over-documenting without measuring, ignoring A.5 until the audit, assuming 27001 covers the AI part, thinking certification is a one-off project rather than a living system).

The analogy: the aircraft operations manual

A modern aircraft, an A350, a Boeing 787, a drone certified for urban delivery, does not fly because it has good engines. It flies because the organisation that operates it has an Operations Manual approved by the aviation authority (EASA in Europe, the FAA in the US, AESA in Spain as a delegated body). The manual does not contain the engine blueprints, which the manufacturer certifies; it contains the procedures: who is captain on each flight, which checklist is run before each take-off, which periodic inspections happen at 100, 500 and 2,000 flight hours, which external suppliers are authorised to touch which components, what is documented after each incident, what to do when a new alert appears on the panel. The authority does not sit in on every flight: it reads the manual, randomly audits the traceability of past flights against the manual, and if everything adds up, it keeps the certification in place.

An AI system in production, the multi-tenant chatbot from the forensic post, a copilot for lawyers, a credit scoring system, is exactly the same. It flies because the model is good, the LLMOps pipeline is well built, the guardrails catch the bad cases. But it certifies because the organisation operating it has an AIMS (AI Management System) described in an auditable manual. ISO/IEC 42001 is that manual: its mandatory table of contents (Annex SL, seven clauses) and its catalogue of AI-specific controls (Annex A, 38 controls). The auditor does not sit next to the MLOps engineer: they read the AI policy, review the impact assessments of the most recently deployed systems, check that incident retrain is documented, verify third-party contracts, audit a sample of Langfuse traces cross-checked against dataset_hash and prompt_id. And if everything adds up, they certify.

The analogy matters because it frames the right question: 42001 does not certify the model or the code. It certifies the way of operating the complete system. A team can have the best OSS stack in the world and fail the audit because it has no written AI policy and no documented decision about which role (provider vs producer vs customer) it occupies towards its clients. And the reverse: a team with a modest model but with operations-manual discipline can certify without acrobatics.

ISO/IEC 42001 in 15 seconds

Publication: December 2023, ISO/IEC JTC 1/SC 42 (the ISO/IEC subcommittee for AI).
Status in 2026: an active standard, certifiable by accredited bodies (BSI, AENOR, TÜV, Bureau Veritas, A-LIGN, Schellman). Not yet formally recognised as a harmonised standard under the EU AI Act, but it provides the management base to lean on.
Compatibility: it shares the Annex SL structure with ISO 9001 (quality), 27001 (information security), 27701 (privacy), 22301 (continuity), 20000-1 (IT services). Organisations with integrated management systems (IMS) add it with 20-40% of the effort it would take to implement from scratch.
Applicability: any organisation that develops, provides, deploys or uses AI systems. It is not limited to developers: a company that consumes a hosted LLM and integrates it into its own product is within scope.
Certification: a 3-year cycle with an initial audit (Stage 1: documentation review + Stage 2: on-site audit) and annual surveillance audits. Typical cost: 15,000-60,000 € for the initial one depending on size; 6,000-20,000 € per annual surveillance.

What 42001 does not do:

It does not say which models to use or which thresholds to apply.
It does not certify the individual model (that is done by specific evaluations such as a NIST AI RMF profile or EU AI Act technical documentation).
It does not replace the EU AI Act or the GDPR: it is complementary. Implementing it well makes legal compliance easier but does not guarantee it.
It is not a technical standard for explainability or robustness (those are ISO/IEC 25059, 24029, 23894 and others in the SC 42 family).

Distinction from neighbouring frameworks

Framework	Nature	Scope	Certifiable	Overlap with 42001
ISO/IEC 42001:2023	Management standard	AIMS for any AI system	Yes	—
EU AI Act (Reg. 2024/1689)	Binding legal regulation	AI systems in the EU, risk-categorised	No (it is law)	Arts 9, 10, 11, 12, 13, 14, 17
NIS2 (Dir. 2022/2555)	Cybersecurity directive	Essential/important entities	Via the national scheme	Asset register, incident, supply chain
ENS (RD 311/2022)	Spanish security regulation	Public sector and its suppliers	Yes (B/M/A categories)	Traceability, incident management
ISO/IEC 27001	Management standard	Information security	Yes	Annex SL structure + Annex A overlap
ISO/IEC 27701	Management standard	Privacy (extends 27001)	Yes	PII in training data
NIST AI RMF 1.0	Voluntary framework	AI risk management	No	Conceptually aligned, not identical
ISO/IEC 23894	Technical standard	AI risk management	No	Input to A.5 (impact assessment)
ISO/IEC 5259	Family	Data quality for AI	No	Input to A.7 (data)

Three distinctions that matter operationally and are a recurring source of confusion with clients:

ISO 42001 ≠ EU AI Act compliance. Holding the 42001 certification makes it easier to demonstrate articles 9-17 of the European Regulation, but the Regulation requires things 42001 does not cover directly (CE marking of high-risk systems, registration in the European database, declaration of conformity, specific post-market monitoring). Implementing 42001 first and then filling the AI Act gaps is the standard route.
ISO 27001 is not enough. 27001 covers confidentiality, integrity and availability of information. The AI side is missing: bias, opacity, model drift, training corpus quality, human evaluation, impact on affected persons. 42001 is a complement, not a substitute. Organisations that already have 27001 in place have an advantage because they share half the documentation.
NIS2 ≠ AI safety. NIS2 requires registering critical assets, notifying incidents within 24 h, and managing the digital supply chain. AI systems can be within NIS2 scope if they form part of the critical asset (an LLM serving customer support in a financial entity is), but NIS2 does not audit model quality. 42001 does.

The seven clauses (Annex SL): the mandatory table of contents

The seven clauses, from clause 4 to clause 10, are common to all modern management standards (Annex SL, also called the “High Level Structure”). That means an organisation with ISO 9001 or 27001 already in place recognises the structure. Clauses 1-3 are introductory (scope, normative references, terms).

Clause 4 — Context of the organisation

Identify the external context (applicable regulation, customer expectations, social risks) and the internal context (strategy, culture, capabilities). Identify the interested parties and their expectations: customers, regulators, affected persons, employees, suppliers. Define the scope of the AIMS: which AI systems are in and which are out.

The usual gap: organisations that say “all our AI systems are in scope” without having enumerated them. The auditor asks for the list. Without a list, there is no scope.

Clause 5 — Leadership

Top management must approve and publish an AI policy, assign roles and responsibilities (typically AI lead, AI risk owner, data officer), and demonstrate commitment through resources, communication and oversight. The policy is an auditable document and must be made available to staff and interested parties.

The usual gap: a generic AI policy copied off the internet, with no measurables and no concrete objectives. The auditor asks how compliance with it is measured. Without metrics, the policy is theatre.

Clause 6 — Planning

Identify risks and opportunities of the AIMS (not of the individual model). Define measurable AI objectives, with deadlines and owners. Plan changes to the AIMS.

The usual gap: confusing AIMS risks (what happens if we do not document correctly?) with model risks (what happens if the model is biased?). The first belongs here; the second goes to A.5.

Clause 7 — Support

Resources: human, technical, financial, infrastructure. Competence of personnel (documented training). Awareness of the policy among personnel. Internal and external communication. Documented information (the backbone of the management system: policy, procedures, records, evidence).

The usual gap: documentation scattered across Confluence/Notion/Drive with no version control and no recorded approvals. The auditor asks about the last change: who approved it? when? on what grounds?

Clause 8 — Operation

The most operational clause. It requires:

Operational planning and control: how the AI system’s lifecycle is managed day to day. → Covered in the blog by the six-stage LLMOps pipeline.
Impact assessment (linked to A.5).
AI system lifecycle management (linked to A.6).
Data for AI systems (linked to A.7).

This is the clause that materialises in controls A.5, A.6 and A.7. On its own it adds no new requirements: it links to Annex A.

Clause 9 — Performance evaluation

Monitoring, measurement, analysis, evaluation. Internal audits (planned, with criteria, scope, frequency, recorded results). Management review (typically quarterly or half-yearly, with a mandatory agenda: inputs, evidence, decisions, actions).

The usual gap: there is OTel tracing + Langfuse + Grafana and plenty of data, but there is no formal management review agenda with documented minutes. The auditor asks for the minutes. Without minutes, there is no review.

Clause 10 — Improvement

Nonconformity and corrective action: when something fails, it is recorded, root cause is analysed, a correction is agreed, effectiveness is verified. Continual improvement: the system evolves deliberately.

The usual gap: Jira tickets with technical post-mortems but no formal “ISO nonconformity” record closed out with an effectiveness check. They are two distinct artefacts, even if they can be integrated.

The 38 Annex A controls: the AI-specific catalogue

Unlike Annex SL (which is common), Annex A is 42001’s AI-specific signature. The 38 controls are organised into 9 sections (A.2 to A.10; A.1 is the introduction) covering the AI-specific risks: opacity, bias, drift, corpus quality, impact on affected persons, third-party dependency. Each control has an objective (what it aims to achieve) and implementation guidance in Annex B.

Section	Focus	# controls
A.2	AI-related policies	2
A.3	Internal organisation	3
A.4	Resources for AI systems	6
A.5	Impact assessment	5
A.6	AI system lifecycle	4
A.7	Data for AI systems	5
A.8	Information for interested parties	4
A.9	Use of AI systems	3
A.10	Third parties and customer relationships	4
Total	—	38

What follows is the section-by-section mapping of the controls onto material we have already covered in the blog. The editorial intent is to show which gaps remain once the technical architecture is in place, so that the road to certification does not start from zero.

Cross-mapping: 38 controls ↔ blog posts

A.2 — AI policies (2 controls): PARTIAL

A.2.2 AI policy: the organisation must have a documented AI policy, approved by management, reviewed periodically, communicated and available. It covers principles, scope and commitments.
A.2.3 Alignment with other policies: the AI policy is not an orphan, it aligns with existing security, privacy, quality and ethics policies.

Gap: this is not a code matter. The AI policy is a document that the organisation’s management approves and signs. The blog teaches a neutral, technical editorial stance (no hype, sovereignty, OSS by default under ENS/NIS2), but that is not any specific organisation’s AI policy. Each client has to draft and sign their own.

Minimum template: 1-2 pages with: principles (transparency, human oversight, fairness, accountability, sustainability), scope (which systems), measurable commitments (annual review, impact assessment before deployment, team training), governance (who approves what).

A.3 — Internal organisation (3 controls): GAP

A.3.2 AI roles and responsibilities: roles defined, non-overlapping, communicated. Typically: AI lead, AI risk owner, data steward, AI ethics officer (which may be a single person in small organisations).
A.3.3 Reporting of AI incidents/concerns: a channel for anyone (internal or external) to report a problem with an AI system, with documented follow-up.
A.3.4 Identification of stakeholders: a maintained list of stakeholders (customers, affected persons, regulators, partners) and their expectations.

Gap: also not technical. An organisational decision. The usual way to cover it is to appoint an AI lead (which may be the CIO, the CTO or a new role depending on size), reuse the security reporting channel (typically already in place because of 27001) by extending it to AI, and maintain a live stakeholder register.

A.4 — Resources (6 controls): COVERED

A.4.2 Documented information: documentation of the AIMS.
A.4.3 Data resources: identification and management of the data available for training, evaluation and operation.
A.4.4 Tooling resources: development, validation and monitoring tools.
A.4.5 System resources: hardware, infrastructure, compute.
A.4.6 Human resources: competent personnel.
A.4.7 Financial resources: budget.

Covered by the blog in the three architectural posts:

Anatomy of the stack: seven layers: A.4.5 system resources, A.4.4 tooling.
Seven deployment phases: A.4.5 + A.4.7 (budget implicitly).
Five maturity levels: A.4.5 + A.4.6 (team maturity).
OSS tooling catalogue: A.4.4 tooling.
Data versioning with DVC and lakeFS: A.4.3 data resources.

A.5 — Impact assessment (5 controls): PARTIAL

A.5.2 AI impact assessment process: a documented impact assessment procedure.
A.5.3 Documentation of AI impact assessments: records of the assessments carried out.
A.5.4 Alignment with AI risk treatment: the impact assessment’s decisions feed risk treatment.
A.5.5 Impacts on individuals: specific dimensions covering affected persons (rights, discrimination, privacy).
A.5.6 Societal impacts: dimensions covering society (information, social rights).

Partial: the method exists in the ISO/IEC SC 42 family. ISO/IEC 23894:2023 is the technical standard for AI risk management and NIST AI RMF 1.0 is the freely usable American equivalent. But the organisation has to write its procedure and run it per system before deployment. It is not code, it is discipline.

Minimum template for the impact assessment (3-5 pages per system):

Description of the system (what it does, who it serves, underlying model and stack).
Stakeholders identified.
Potential impacts (intended and unintended) on people, groups and society.
Fairness and robustness metrics applied, with thresholds and results.
Mitigations applied (guardrails, evals, human oversight, rate limiting).
Accepted residual risks, with signed justification.
Review cadence (typically annual, or on substantial change).

A.6 — AI system lifecycle (4 controls): COVERED

A.6.2.2 Objectives for responsible development of AI: responsible development objectives defined per system.
A.6.2.3 Processes for responsible AI design and development: design and development procedures.
A.6.2.4 AI system requirements and specifications: formal specification of the system.
A.6.2.5 Verification and validation: V&V before and during operation.
A.6.2.6 Deployment: deployment procedures.
A.6.2.7 Operation and monitoring: continuous operation and monitoring.
A.6.2.8 Documentation: lifecycle documentation.

Covered by the blog:

The six-stage LLMOps pipeline: the complete master map of the lifecycle.
Anatomy of an LLM request: the forensic version of how it runs in production.
Continuous fine-tuning in production: the A.6.2.3 + A.6.2.5 + A.6.2.6 + A.6.2.7 discipline in real operation.
Evals: the layer after tracing: A.6.2.5 verification and validation.
Retrain: A.6.2.7 operation + continuous iteration.
Modern alignment: DPO, KTO, ORPO, SimPO: A.6.2.3 responsible design.
MLOps landscape 2026: the tooling landscape.

A.7 — Data for AI systems (5 controls): COVERED

A.7.2 Data for development and enhancement of AI: data management policy and procedures for development and improvement.
A.7.3 Acquisition of data: acquisition procedures (origin, authorisation, quality).
A.7.4 Quality of data for AI systems: measurable quality criteria.
A.7.5 Data provenance: data lineage.
A.7.6 Data preparation: preparation procedures (chunking, anonymisation, labelling).

Covered by the blog:

RAG corpus curation: the active librarian: A.7.4 + A.7.5 + A.7.6 in detail (five layers: schema, dedup, PII, anti-contamination, lineage).
Data versioning: DVC and lakeFS: A.7.2 + A.7.5 (the four data artefacts versioned with lineage).
Reranker and hybrid retrieval: A.7.6 preparation + filtering.
LLM Guard: A.7.6 runtime anonymisation with Vault.

A.8 — Information for interested parties (4 controls): COVERED

A.8.2 System documentation and information for users: technical documentation available.
A.8.3 External reporting: the capacity to report to authorities where applicable.
A.8.4 Communication of incidents to users: notification to users when there is an incident.
A.8.5 Information for interested parties: information for other stakeholders.

Covered by the blog:

LLM tracing with OpenTelemetry GenAI: A.8.2 per-request traceability, A.8.3 the capacity to extract forensic reporting.
Prompt versioning with Langfuse and MLflow: A.8.2 documented versioning.
Guardrails and safety in LLMs: A.8.4, gen_ai.guardrail.* spans as the basis for incident notification.
LLM Guard: A.8.4 incident events for retrain.

A.9 — Use of AI systems (3 controls): COVERED

A.9.2 Processes for responsible use of AI: responsible use procedures.
A.9.3 Objectives for responsible use of AI: objectives.
A.9.4 Intended use of AI systems: documentation of the intended use.

Covered by the blog:

Guardrails and safety in LLMs: A.9.2 + A.9.3 (the four lines of defence).
LLM Guard: A.9.2 operational detail.
Evals: the layer after tracing: A.9.3 measurement of objectives.
Retrain: A.9.2 closed loop.

A.10 — Third parties and customer relationships (4 controls): COVERED

A.10.2 Allocation of responsibilities: distribution of responsibilities between AI roles.
A.10.3 Suppliers: procedures for AI suppliers.
A.10.4 Customers: procedures towards customers.
A.10.5 Third parties: procedures for third parties.

Covered by the blog:

The parallel catalogue: OSS vs hyperscalers: A.10.3 supplier assessment with lock-in and contractual sovereignty analysis.
The OSS catalogue for LLMOps: A.10.5 third-party inventory (OSS components with licences and governance).
Anatomy of an LLM request: A.10.2 + A.10.4 in the multi-tenant case.

The roles defined by the standard

ISO/IEC 22989:2022 (AI vocabulary, complementary to 42001) defines six roles. Each organisation has to decide which ones it occupies and document it:

Role	Definition	Main responsibility	Example
AI provider	Organisation that provides the AI system to others	Makes the system available	OpenAI provides GPT-5 via API
AI producer	Organisation that develops the AI system	Design, development, validation	Meta produces Llama 4
AI customer	Organisation that acquires the AI system	Selection, integration, oversight	A consultancy integrating an LLM into its own product
AI partner	Organisation collaborating with another AI role	Shared	A GPU hardware manufacturer
AI subject	Person or group affected by the system	Recipient of the impact	The chatbot’s end user
Relevant authority	Regulator with jurisdiction	External oversight	AEPD, CNMC, EU AI Act authorities

An organisation can occupy several roles at once, which changes the applicable controls. A common pattern in consultancy is: producer + customer + provider towards the end client. The A.10 responsibilities are modulated by the roles.

Example role mapping for the multi-tenant chatbot from the forensic post:

Base model manufacturer (Llama 4): AI producer of the base model.
OSS stack operator (the consultancy): AI producer of the LoRA adapter + AI provider of the chatbot to its clients + AI customer of Meta’s base model.
End client (the insurer): AI customer of the chatbot + AI provider of the customer support service.
The policyholder: AI subject.
AEPD + EU AI Act authority: relevant authority.

Each box in the table generates different obligations. The consultancy, being producer of the adapter, must document A.6 (lifecycle) and A.7 (data) for the adapter. Being provider of the chatbot, it must document A.10.4 (customers). Being customer of the base model, it must document A.10.3 (suppliers) and validate that Meta does its part.

Impact levels and proportionality

42001 does not demand the same rigour of every system. Clause 6.1.2 and control A.5 introduce impact as a modulator. The standard does not define strict categories (unlike the EU AI Act, which does define “prohibited / high risk / limited risk / minimal”), but it recommends using levels according to severity and probability.

Industry practice in 2026 aligns the 42001 levels with the EU AI Act categories:

42001 level	EU AI Act	Examples	Control depth
High	High risk (Annex III)	Credit scoring, HR, health, critical infrastructure	Exhaustive impact assessment, mandatory human oversight, continuous monitoring, adversarial evals, detailed logging, half-yearly management review
Medium	Limited risk	Customer service chatbots that do not automate decisions, productivity assistants	Standard impact assessment, complete guardrails, annual review
Low	Minimal risk	Spam filters, non-personalised content recommendations	Lightweight impact assessment, basic controls

This proportionality is operationally key: implementing 42001 at maximum rigour for a low-risk system is waste; relaxing it on a high-risk one is non-compliance.

The seven minimum AIMS documents

An auditor at Stage 1 (documentation review) asks for between seven and ten documents. The seven essentials:

AI policy (clause 5.2 + A.2.2). 1-2 pages. Approved by management, dated, versioned.
AIMS scope (clause 4.3). List of AI systems in scope, inclusion criteria.
Stakeholder register (clause 4.2 + A.3.4). A maintained list with expectations.
AIMS risk register (clause 6.1). Risks of the management system, not of each model.
Impact assessment procedure (A.5.2) + records of assessments carried out (A.5.3). The procedure plus one or more assessments done.
AI lifecycle procedure (A.6.2), which can literally read “see the six-stage LLMOps pipeline” with references to technical runbooks.
Data management procedure (A.7.2), covering acquisition, quality, provenance, preparation, anonymisation.

Common additional documents:

Responsible use policy (A.9.2) with permitted and non-permitted use types.
AI third-party management procedure (A.10.3, A.10.5) with assessment criteria for AI suppliers.
Internal audit plan + management review agenda (clauses 9.2 + 9.3).

For an organisation with a mature OSS stack, documents 6 and 7 are references to technical artefacts that already exist (pipeline runbooks, DVC configurations, the PII policy in LLM Guard). The real documentary effort sits in documents 1, 2, 3, 4 and 5.

Applied case: the blog’s multi-tenant chatbot → 42001 checklist

We take the system described in the forensic post, a multi-tenant customer support chatbot for insurers on an on-premise OSS stack, and walk it as a 42001 auditor would.

Clause 4 — Context. The AIMS scope includes the chatbot and excludes the internal HR system (a different AI). Stakeholders identified: client insurers, affected policyholders, the AEPD, the EU AI Act authority (once it enters into force on 2 Aug 2026), the supplier Meta (base model), the hardware supplier NVIDIA. → Documented.

Clause 5 — Leadership. AI policy signed by the CEO, in force. Roles assigned: AI lead (CTO), AI risk owner (CISO), data steward (Head of Data), quarterly AI ethics committee. → Documented.

Clause 6 — Planning. AIMS risk register: incomplete documentation, team churn, dependency on a single GPU supplier, EU AI Act regulatory change. AIMS objectives for 2026: 42001 certification before Q4, EU AI Act high-risk compliance before 2 Aug. → Documented.

Clause 7 — Support. Resources: a 4×H100 SXM cluster + the seven stack layers. Competence: 2 MLE + 2 SRE + 1 part-time AI ethics, all with documented training. Communication: AI policy on the intranet + handbook. → Documented.

Clause 8 — Operation. Operating procedures = the six-stage LLMOps pipeline. Impact assessment run before deployment + annual review + review on substantial change (defined as: base model change, major adapter change, expansion to a new tenant). → Documented.

Clause 9 — Evaluation. Monitoring: Langfuse + Tempo + VictoriaMetrics + Grafana. Mandatory dashboard metrics: F1 per guardrail category over real traffic, statistical drift, RAG faithfulness, refusal rate. Quarterly internal audit with written criteria. Half-yearly management review with signed minutes. → Documented.

Clause 10 — Improvement. Incident-driven retrain tickets mapped as nonconformities when severity ≥ HIGH. Documented root cause analysis. Effectiveness verified at the next eval gate. → Documented.

Annex A — By section:

A.2 (Policies): AI policy + responsible use policy. → Documented.
A.3 (Organisation): roles assigned, reporting channel, stakeholder register. → Documented.
A.4 (Resources): seven deployment phases + OSS catalogue + training plan + annual budget. → Documented.
A.5 (Impact): procedure + per-system assessments + fairness metrics applied. → Documented.
A.6 (Lifecycle): LLMOps pipeline + continuous fine-tuning + retrain. → Documented.
A.7 (Data): data versioning + RAG corpus curation + LLM Guard Vault + Presidio. → Documented.
A.8 (Information to parties): OTel tracing + Langfuse + gen_ai.guardrail.* spans + notification to tenants within SLA. → Documented.
A.9 (Use): guardrails + evals + responsible use policy. → Documented.
A.10 (Third parties): OSS vs hyperscalers with lock-in analysis + Meta contract for the base model + contracts with tenants. → Documented.

Result of the walkthrough: certifiable. The typical gaps (A.2.2 written policy, A.3 roles, A.5 impact assessment procedure) are covered as formal documents. The operational clauses (8, 9, 10) lean on the blog’s technical architecture. The distance between “having the architecture” and “having certification” is measured in documentary discipline, not in code.

Cross-mapping with the EU AI Act, NIS2 and ENS

EU AI Act (Reg. 2024/1689) — seven directly aligned articles

EU AI Act article	Topic	Aligned 42001 control	Applies to
Art. 9	Risk management system	A.5 + clause 6	High-risk systems
Art. 10	Data and data governance	A.7 (all)	High-risk systems
Art. 11	Technical documentation	A.6 + A.4.2	High-risk systems
Art. 12	Record-keeping (logs)	A.8.2 + OTel tracing	High-risk systems
Art. 13	Transparency to deployers	A.8.5 + A.10.4	High-risk systems
Art. 14	Human oversight	A.9.2 + documented oversight	High-risk systems
Art. 17	Quality management system	Clauses 4-10	High-risk providers

The main obligations for high-risk systems enter into application on 2 August 2026. Implementing 42001 now builds the management base that deadline demands.

What is missing for EU AI Act compliance that 42001 does not cover:

CE conformity of high-risk systems (declaration of conformity, marking, registration in the EU database).
The specific post-market monitoring required by Art. 72.
Serious incident reporting to authorities within legal deadlines (not only to users).
Transparency obligations towards users for limited-risk systems (Art. 50): chatbots, deepfakes, generated content.
The Art. 5 prohibitions (social scoring, manipulation, real-time biometrics with exceptions).

NIS2 (Dir. 2022/2555) — three pillars with overlap

Asset register (Art. 21.2.f): AI systems within NIS2 scope must be in the asset inventory. → Overlaps with A.4 + clause 4.3 (scope).
Incident notification (Art. 23): significant incidents are notified within 24 h (initial alert) + 72 h (detailed report). → Overlaps with A.3.3 (reporting) + clause 10 (improvement).
Supply chain security (Art. 21.2.d): security assessment of the digital supply chain. → Overlaps with A.10.3 (suppliers).

For essential NIS2 entities that also use AI systems, 42001 covers the AI-specific part that NIS2 requires by inference but does not spell out.

ENS (RD 311/2022)

The Spanish Esquema Nacional de Seguridad already expressly addresses AI in its Annex II (ENS controls). The Basic/Medium/High categories align with the 42001 impact levels. The ENS controls on traceability (op.exp.8), activity logging (op.exp.10) and incident management (op.exp.7) are covered by the same technical artefacts as A.8 and A.5 of 42001. An organisation certified at ENS High category with AI systems is a reasonable effort away from adding 42001.

The five usual certification traps

Trap 1 — Confusing 42001 with EU AI Act compliance. Passing the 42001 audit does not imply conformity with the European Regulation. They are distinct universes with a 60-70% overlap. The trap surfaces when the client asks for CE marking of the high-risk system and the organisation produces only the 42001 certificate.

Trap 2 — Over-documenting. Two-hundred-page manuals with procedures copied from templates, with no measurables and no evidence of application. The auditor asks for the last execution of the procedure; with no records, the procedures are ornament. The practical rule: prefer short documents referencing live technical artefacts to long self-contained ones.

Trap 3 — Under-measuring. Defining AIMS objectives with no operational metrics. “Improve model quality” is a null objective; “F1 per guardrail category ≥ 0.85 over real traffic, measured weekly, reviewed quarterly in the management review” is an auditable objective. The blog has insisted on this in every post on evals, guardrails and retrain.

Trap 4 — Ignoring A.5 until audit day. The impact assessment is the most underrated control and the first one the auditor asks for. Without per-system assessments carried out before deployment, there is no way to demonstrate A.5. The trap surfaces when there is no longer time to produce credible retrospective assessments.

Trap 5 — Assuming 27001 covers the AI part. Organisations that already have 27001 sometimes think “we have half of it done”. That is true for Annex SL (structure) and for A.5/A.6/A.7 of 27001 (not of 42001) as far as infosec goes. It is false for A.5 of 42001 (impact assessment), A.7 of 42001 (AI-specific data quality), A.9 (responsible use) and A.10.4 (AI customers). You have to add, not assume.

What we have not covered (upcoming posts)

Concrete templates of the seven mandatory documents, with drafting examples and metrics. Material for a post along the lines of “The AIMS manual in 7 documents” with a reference frame.
Detailed article-by-article mapping to the EU AI Act with the checklist of technical evidence derivable from the blog’s OSS stack. Especially Arts 11 (technical documentation), 14 (human oversight) and 72 (post-market monitoring).
The combined ENS High category + 42001 case: which ENS controls are covered by which AIMS artefacts, avoiding duplication.
NIST AI RMF 1.0 vs 42001 comparison: many international clients ask for both. How the same artefacts are recycled to satisfy the two frameworks.
42001 for LLM agents and MCP: the new dimensions that emerge when the AI system is agentic (excessive agency, tool use, graduated autonomy). The guardrails post introduced line 3 (tool GR); 42001 has open gaps in this terrain and SC 42 is working on addenda.

References

ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system. ISO. https://www.iso.org/standard/81230.html.
ISO/IEC 22989:2022 — Information technology — Artificial intelligence — Artificial intelligence concepts and terminology. Define los roles AI provider/producer/customer/partner/subject.
ISO/IEC 23894:2023 — Information technology — Artificial intelligence — Guidance on risk management. Insumo de A.5.
ISO/IEC 38507:2022 — Governance implications of the use of AI by organizations. Complemento de gobierno.
ISO/IEC 5259 — Data quality for analytics and machine learning (familia). Insumo de A.7.
EU AI Act (Regulation 2024/1689) — texto consolidado en EUR-Lex. Entrada en vigor de obligaciones de alto riesgo: 2 ago 2026.
NIS2 (Directive 2022/2555) — texto consolidado en EUR-Lex.
ENS — Real Decreto 311/2022 — Esquema Nacional de Seguridad, BOE-A-2022-7191.
NIST AI RMF 1.0 (2023) — https://www.nist.gov/itl/ai-risk-management-framework.
EUR-Lex EU AI Act consolidated text — https://eur-lex.europa.eu/eli/reg/2024/1689.
A-LIGN / BSI / Schellman — blogs sobre experiencia de auditoría 42001 con casos reales 2024-2025.

Audit on lo0 — Tech Blog

Virtual keys, budgets and limits in LiteLLM: the layer that decides who gets the GPU, and the four things the documentation gets wrong

TL;DR

You are here: the gateway as an administrative boundary

The analogy: the building’s key ring

The hierarchy, and the line that breaks it

Virtual keys: the complete life cycle

How they are stored

Rotation

What exactly happens when a budget runs out

Capacity limits: the v3 limiter

What the client sees

Identity: beyond the static key

Audit: what gets recorded

What this demonstrates to an auditor, and what it does not

Checklist

Traps and things that are not what they look like

Closing

See also

Sources

ISO/IEC 42001: the operations manual for the AI system — how the AIMS fits over the on-premise LLM platform described in this blog

TL;DR

The analogy: the aircraft operations manual

ISO/IEC 42001 in 15 seconds

Distinction from neighbouring frameworks

The seven clauses (Annex SL): the mandatory table of contents

Clause 4 — Context of the organisation

Clause 5 — Leadership

Clause 6 — Planning

Clause 7 — Support

Clause 8 — Operation

Clause 9 — Performance evaluation

Clause 10 — Improvement

The 38 Annex A controls: the AI-specific catalogue

Cross-mapping: 38 controls ↔ blog posts

A.2 — AI policies (2 controls): PARTIAL

A.3 — Internal organisation (3 controls): GAP

A.4 — Resources (6 controls): COVERED

A.5 — Impact assessment (5 controls): PARTIAL

A.6 — AI system lifecycle (4 controls): COVERED

A.7 — Data for AI systems (5 controls): COVERED

A.8 — Information for interested parties (4 controls): COVERED

A.9 — Use of AI systems (3 controls): COVERED

A.10 — Third parties and customer relationships (4 controls): COVERED

The roles defined by the standard

Impact levels and proportionality

The seven minimum AIMS documents

Applied case: the blog’s multi-tenant chatbot → 42001 checklist

Cross-mapping with the EU AI Act, NIS2 and ENS

EU AI Act (Reg. 2024/1689) — seven directly aligned articles

NIS2 (Dir. 2022/2555) — three pillars with overlap

ENS (RD 311/2022)

The five usual certification traps

What we have not covered (upcoming posts)

References

See also