Lineage on lo0 — Tech Blog

ISO/IEC 42001: the operations manual for the AI system — how the AIMS fits over the on-premise LLM platform described in this blog

Mon, 01 Jun 2026 06:00:00 +0200

This post closes an asymmetry the blog had been accumulating: we have described in detail the technical platform (seven stack layers, seven deployment phases, five maturity levels), the operational pipeline (six LLMOps stages), the data pieces (corpus curation, versioning), the eval / safety pieces (evals, guardrails, LLM Guard) and the observe pieces (OTel GenAI tracing). What had not appeared is the governance layer a regulated client asks for on top of all that. ISO/IEC 42001 is that layer.

TL;DR

ISO/IEC 42001:2023 is the first certifiable international standard that defines how an AI system is managed. It is not a technical standard (it does not say “use this inference engine” or “this safety threshold”): it is a management standard, a cousin of ISO 27001 and ISO 9001. From both it inherits the Annex SL structure, seven mandatory clauses running through context, leadership, planning, support, operation, performance evaluation and improvement, and it adds an Annex A with 38 AI-specific controls in 9 sections: policies, internal organisation, resources, impact assessment, lifecycle, data, information to interested parties, use, third parties. The thesis of this post is that the technical architecture described in this blog directly covers between 60% and 80% of the A controls with no extra work. The LLMOps pipeline materialises A.6, versioning and curation materialise A.7, guardrails and evals materialise A.9, OTel tracing materialises A.8. The rest is governance discipline that does not appear in the code (a written AI policy, impact assessments per system, a stakeholder register, role decisions between provider/producer/customer, mandatory documentation), and that is precisely what separates a real certification from performative compliance. The post maps the correspondence control by control, cross-references it with the EU AI Act (seven articles directly aligned with 42001: 9, 10, 11, 12, 13, 14, 17), with NIS2 (asset register, incident notification, supply chain) and with ENS (Esquema Nacional de Seguridad, Spain’s national security framework, RD 311/2022, Basic/Medium/High categories), lists the seven mandatory minimum documents an auditor asks for, presents the blog’s multi-tenant chatbot as a live 42001 checklist, and closes with the five usual traps (confusing 42001 with EU AI Act compliance, over-documenting without measuring, ignoring A.5 until the audit, assuming 27001 covers the AI part, thinking certification is a one-off project rather than a living system).

The analogy: the aircraft operations manual

A modern aircraft, an A350, a Boeing 787, a drone certified for urban delivery, does not fly because it has good engines. It flies because the organisation that operates it has an Operations Manual approved by the aviation authority (EASA in Europe, the FAA in the US, AESA in Spain as a delegated body). The manual does not contain the engine blueprints, which the manufacturer certifies; it contains the procedures: who is captain on each flight, which checklist is run before each take-off, which periodic inspections happen at 100, 500 and 2,000 flight hours, which external suppliers are authorised to touch which components, what is documented after each incident, what to do when a new alert appears on the panel. The authority does not sit in on every flight: it reads the manual, randomly audits the traceability of past flights against the manual, and if everything adds up, it keeps the certification in place.

An AI system in production, the multi-tenant chatbot from the forensic post, a copilot for lawyers, a credit scoring system, is exactly the same. It flies because the model is good, the LLMOps pipeline is well built, the guardrails catch the bad cases. But it certifies because the organisation operating it has an AIMS (AI Management System) described in an auditable manual. ISO/IEC 42001 is that manual: its mandatory table of contents (Annex SL, seven clauses) and its catalogue of AI-specific controls (Annex A, 38 controls). The auditor does not sit next to the MLOps engineer: they read the AI policy, review the impact assessments of the most recently deployed systems, check that incident retrain is documented, verify third-party contracts, audit a sample of Langfuse traces cross-checked against dataset_hash and prompt_id. And if everything adds up, they certify.

The analogy matters because it frames the right question: 42001 does not certify the model or the code. It certifies the way of operating the complete system. A team can have the best OSS stack in the world and fail the audit because it has no written AI policy and no documented decision about which role (provider vs producer vs customer) it occupies towards its clients. And the reverse: a team with a modest model but with operations-manual discipline can certify without acrobatics.

ISO/IEC 42001 in 15 seconds

Publication: December 2023, ISO/IEC JTC 1/SC 42 (the ISO/IEC subcommittee for AI).
Status in 2026: an active standard, certifiable by accredited bodies (BSI, AENOR, TÜV, Bureau Veritas, A-LIGN, Schellman). Not yet formally recognised as a harmonised standard under the EU AI Act, but it provides the management base to lean on.
Compatibility: it shares the Annex SL structure with ISO 9001 (quality), 27001 (information security), 27701 (privacy), 22301 (continuity), 20000-1 (IT services). Organisations with integrated management systems (IMS) add it with 20-40% of the effort it would take to implement from scratch.
Applicability: any organisation that develops, provides, deploys or uses AI systems. It is not limited to developers: a company that consumes a hosted LLM and integrates it into its own product is within scope.
Certification: a 3-year cycle with an initial audit (Stage 1: documentation review + Stage 2: on-site audit) and annual surveillance audits. Typical cost: 15,000-60,000 € for the initial one depending on size; 6,000-20,000 € per annual surveillance.

What 42001 does not do:

It does not say which models to use or which thresholds to apply.
It does not certify the individual model (that is done by specific evaluations such as a NIST AI RMF profile or EU AI Act technical documentation).
It does not replace the EU AI Act or the GDPR: it is complementary. Implementing it well makes legal compliance easier but does not guarantee it.
It is not a technical standard for explainability or robustness (those are ISO/IEC 25059, 24029, 23894 and others in the SC 42 family).

Distinction from neighbouring frameworks

Framework	Nature	Scope	Certifiable	Overlap with 42001
ISO/IEC 42001:2023	Management standard	AIMS for any AI system	Yes	—
EU AI Act (Reg. 2024/1689)	Binding legal regulation	AI systems in the EU, risk-categorised	No (it is law)	Arts 9, 10, 11, 12, 13, 14, 17
NIS2 (Dir. 2022/2555)	Cybersecurity directive	Essential/important entities	Via the national scheme	Asset register, incident, supply chain
ENS (RD 311/2022)	Spanish security regulation	Public sector and its suppliers	Yes (B/M/A categories)	Traceability, incident management
ISO/IEC 27001	Management standard	Information security	Yes	Annex SL structure + Annex A overlap
ISO/IEC 27701	Management standard	Privacy (extends 27001)	Yes	PII in training data
NIST AI RMF 1.0	Voluntary framework	AI risk management	No	Conceptually aligned, not identical
ISO/IEC 23894	Technical standard	AI risk management	No	Input to A.5 (impact assessment)
ISO/IEC 5259	Family	Data quality for AI	No	Input to A.7 (data)

Three distinctions that matter operationally and are a recurring source of confusion with clients:

ISO 42001 ≠ EU AI Act compliance. Holding the 42001 certification makes it easier to demonstrate articles 9-17 of the European Regulation, but the Regulation requires things 42001 does not cover directly (CE marking of high-risk systems, registration in the European database, declaration of conformity, specific post-market monitoring). Implementing 42001 first and then filling the AI Act gaps is the standard route.
ISO 27001 is not enough. 27001 covers confidentiality, integrity and availability of information. The AI side is missing: bias, opacity, model drift, training corpus quality, human evaluation, impact on affected persons. 42001 is a complement, not a substitute. Organisations that already have 27001 in place have an advantage because they share half the documentation.
NIS2 ≠ AI safety. NIS2 requires registering critical assets, notifying incidents within 24 h, and managing the digital supply chain. AI systems can be within NIS2 scope if they form part of the critical asset (an LLM serving customer support in a financial entity is), but NIS2 does not audit model quality. 42001 does.

The seven clauses (Annex SL): the mandatory table of contents

The seven clauses, from clause 4 to clause 10, are common to all modern management standards (Annex SL, also called the “High Level Structure”). That means an organisation with ISO 9001 or 27001 already in place recognises the structure. Clauses 1-3 are introductory (scope, normative references, terms).

Clause 4 — Context of the organisation

Identify the external context (applicable regulation, customer expectations, social risks) and the internal context (strategy, culture, capabilities). Identify the interested parties and their expectations: customers, regulators, affected persons, employees, suppliers. Define the scope of the AIMS: which AI systems are in and which are out.

The usual gap: organisations that say “all our AI systems are in scope” without having enumerated them. The auditor asks for the list. Without a list, there is no scope.

Clause 5 — Leadership

Top management must approve and publish an AI policy, assign roles and responsibilities (typically AI lead, AI risk owner, data officer), and demonstrate commitment through resources, communication and oversight. The policy is an auditable document and must be made available to staff and interested parties.

The usual gap: a generic AI policy copied off the internet, with no measurables and no concrete objectives. The auditor asks how compliance with it is measured. Without metrics, the policy is theatre.

Clause 6 — Planning

Identify risks and opportunities of the AIMS (not of the individual model). Define measurable AI objectives, with deadlines and owners. Plan changes to the AIMS.

The usual gap: confusing AIMS risks (what happens if we do not document correctly?) with model risks (what happens if the model is biased?). The first belongs here; the second goes to A.5.

Clause 7 — Support

Resources: human, technical, financial, infrastructure. Competence of personnel (documented training). Awareness of the policy among personnel. Internal and external communication. Documented information (the backbone of the management system: policy, procedures, records, evidence).

The usual gap: documentation scattered across Confluence/Notion/Drive with no version control and no recorded approvals. The auditor asks about the last change: who approved it? when? on what grounds?

Clause 8 — Operation

The most operational clause. It requires:

Operational planning and control: how the AI system’s lifecycle is managed day to day. → Covered in the blog by the six-stage LLMOps pipeline.
Impact assessment (linked to A.5).
AI system lifecycle management (linked to A.6).
Data for AI systems (linked to A.7).

This is the clause that materialises in controls A.5, A.6 and A.7. On its own it adds no new requirements: it links to Annex A.

Clause 9 — Performance evaluation

Monitoring, measurement, analysis, evaluation. Internal audits (planned, with criteria, scope, frequency, recorded results). Management review (typically quarterly or half-yearly, with a mandatory agenda: inputs, evidence, decisions, actions).

The usual gap: there is OTel tracing + Langfuse + Grafana and plenty of data, but there is no formal management review agenda with documented minutes. The auditor asks for the minutes. Without minutes, there is no review.

Clause 10 — Improvement

Nonconformity and corrective action: when something fails, it is recorded, root cause is analysed, a correction is agreed, effectiveness is verified. Continual improvement: the system evolves deliberately.

The usual gap: Jira tickets with technical post-mortems but no formal “ISO nonconformity” record closed out with an effectiveness check. They are two distinct artefacts, even if they can be integrated.

The 38 Annex A controls: the AI-specific catalogue

Unlike Annex SL (which is common), Annex A is 42001’s AI-specific signature. The 38 controls are organised into 9 sections (A.2 to A.10; A.1 is the introduction) covering the AI-specific risks: opacity, bias, drift, corpus quality, impact on affected persons, third-party dependency. Each control has an objective (what it aims to achieve) and implementation guidance in Annex B.

Section	Focus	# controls
A.2	AI-related policies	2
A.3	Internal organisation	3
A.4	Resources for AI systems	6
A.5	Impact assessment	5
A.6	AI system lifecycle	4
A.7	Data for AI systems	5
A.8	Information for interested parties	4
A.9	Use of AI systems	3
A.10	Third parties and customer relationships	4
Total	—	38

What follows is the section-by-section mapping of the controls onto material we have already covered in the blog. The editorial intent is to show which gaps remain once the technical architecture is in place, so that the road to certification does not start from zero.

Cross-mapping: 38 controls ↔ blog posts

A.2 — AI policies (2 controls): PARTIAL

A.2.2 AI policy: the organisation must have a documented AI policy, approved by management, reviewed periodically, communicated and available. It covers principles, scope and commitments.
A.2.3 Alignment with other policies: the AI policy is not an orphan, it aligns with existing security, privacy, quality and ethics policies.

Gap: this is not a code matter. The AI policy is a document that the organisation’s management approves and signs. The blog teaches a neutral, technical editorial stance (no hype, sovereignty, OSS by default under ENS/NIS2), but that is not any specific organisation’s AI policy. Each client has to draft and sign their own.

Minimum template: 1-2 pages with: principles (transparency, human oversight, fairness, accountability, sustainability), scope (which systems), measurable commitments (annual review, impact assessment before deployment, team training), governance (who approves what).

A.3 — Internal organisation (3 controls): GAP

A.3.2 AI roles and responsibilities: roles defined, non-overlapping, communicated. Typically: AI lead, AI risk owner, data steward, AI ethics officer (which may be a single person in small organisations).
A.3.3 Reporting of AI incidents/concerns: a channel for anyone (internal or external) to report a problem with an AI system, with documented follow-up.
A.3.4 Identification of stakeholders: a maintained list of stakeholders (customers, affected persons, regulators, partners) and their expectations.

Gap: also not technical. An organisational decision. The usual way to cover it is to appoint an AI lead (which may be the CIO, the CTO or a new role depending on size), reuse the security reporting channel (typically already in place because of 27001) by extending it to AI, and maintain a live stakeholder register.

A.4 — Resources (6 controls): COVERED

A.4.2 Documented information: documentation of the AIMS.
A.4.3 Data resources: identification and management of the data available for training, evaluation and operation.
A.4.4 Tooling resources: development, validation and monitoring tools.
A.4.5 System resources: hardware, infrastructure, compute.
A.4.6 Human resources: competent personnel.
A.4.7 Financial resources: budget.

Covered by the blog in the three architectural posts:

Anatomy of the stack: seven layers: A.4.5 system resources, A.4.4 tooling.
Seven deployment phases: A.4.5 + A.4.7 (budget implicitly).
Five maturity levels: A.4.5 + A.4.6 (team maturity).
OSS tooling catalogue: A.4.4 tooling.
Data versioning with DVC and lakeFS: A.4.3 data resources.

A.5 — Impact assessment (5 controls): PARTIAL

A.5.2 AI impact assessment process: a documented impact assessment procedure.
A.5.3 Documentation of AI impact assessments: records of the assessments carried out.
A.5.4 Alignment with AI risk treatment: the impact assessment’s decisions feed risk treatment.
A.5.5 Impacts on individuals: specific dimensions covering affected persons (rights, discrimination, privacy).
A.5.6 Societal impacts: dimensions covering society (information, social rights).

Partial: the method exists in the ISO/IEC SC 42 family. ISO/IEC 23894:2023 is the technical standard for AI risk management and NIST AI RMF 1.0 is the freely usable American equivalent. But the organisation has to write its procedure and run it per system before deployment. It is not code, it is discipline.

Minimum template for the impact assessment (3-5 pages per system):

Description of the system (what it does, who it serves, underlying model and stack).
Stakeholders identified.
Potential impacts (intended and unintended) on people, groups and society.
Fairness and robustness metrics applied, with thresholds and results.
Mitigations applied (guardrails, evals, human oversight, rate limiting).
Accepted residual risks, with signed justification.
Review cadence (typically annual, or on substantial change).

A.6 — AI system lifecycle (4 controls): COVERED

A.6.2.2 Objectives for responsible development of AI: responsible development objectives defined per system.
A.6.2.3 Processes for responsible AI design and development: design and development procedures.
A.6.2.4 AI system requirements and specifications: formal specification of the system.
A.6.2.5 Verification and validation: V&V before and during operation.
A.6.2.6 Deployment: deployment procedures.
A.6.2.7 Operation and monitoring: continuous operation and monitoring.
A.6.2.8 Documentation: lifecycle documentation.

Covered by the blog:

The six-stage LLMOps pipeline: the complete master map of the lifecycle.
Anatomy of an LLM request: the forensic version of how it runs in production.
Continuous fine-tuning in production: the A.6.2.3 + A.6.2.5 + A.6.2.6 + A.6.2.7 discipline in real operation.
Evals: the layer after tracing: A.6.2.5 verification and validation.
Retrain: A.6.2.7 operation + continuous iteration.
Modern alignment: DPO, KTO, ORPO, SimPO: A.6.2.3 responsible design.
MLOps landscape 2026: the tooling landscape.

A.7 — Data for AI systems (5 controls): COVERED

A.7.2 Data for development and enhancement of AI: data management policy and procedures for development and improvement.
A.7.3 Acquisition of data: acquisition procedures (origin, authorisation, quality).
A.7.4 Quality of data for AI systems: measurable quality criteria.
A.7.5 Data provenance: data lineage.
A.7.6 Data preparation: preparation procedures (chunking, anonymisation, labelling).

Covered by the blog:

RAG corpus curation: the active librarian: A.7.4 + A.7.5 + A.7.6 in detail (five layers: schema, dedup, PII, anti-contamination, lineage).
Data versioning: DVC and lakeFS: A.7.2 + A.7.5 (the four data artefacts versioned with lineage).
Reranker and hybrid retrieval: A.7.6 preparation + filtering.
LLM Guard: A.7.6 runtime anonymisation with Vault.

A.8 — Information for interested parties (4 controls): COVERED

A.8.2 System documentation and information for users: technical documentation available.
A.8.3 External reporting: the capacity to report to authorities where applicable.
A.8.4 Communication of incidents to users: notification to users when there is an incident.
A.8.5 Information for interested parties: information for other stakeholders.

Covered by the blog:

LLM tracing with OpenTelemetry GenAI: A.8.2 per-request traceability, A.8.3 the capacity to extract forensic reporting.
Prompt versioning with Langfuse and MLflow: A.8.2 documented versioning.
Guardrails and safety in LLMs: A.8.4, gen_ai.guardrail.* spans as the basis for incident notification.
LLM Guard: A.8.4 incident events for retrain.

A.9 — Use of AI systems (3 controls): COVERED

A.9.2 Processes for responsible use of AI: responsible use procedures.
A.9.3 Objectives for responsible use of AI: objectives.
A.9.4 Intended use of AI systems: documentation of the intended use.

Covered by the blog:

Guardrails and safety in LLMs: A.9.2 + A.9.3 (the four lines of defence).
LLM Guard: A.9.2 operational detail.
Evals: the layer after tracing: A.9.3 measurement of objectives.
Retrain: A.9.2 closed loop.

A.10 — Third parties and customer relationships (4 controls): COVERED

A.10.2 Allocation of responsibilities: distribution of responsibilities between AI roles.
A.10.3 Suppliers: procedures for AI suppliers.
A.10.4 Customers: procedures towards customers.
A.10.5 Third parties: procedures for third parties.

Covered by the blog:

The parallel catalogue: OSS vs hyperscalers: A.10.3 supplier assessment with lock-in and contractual sovereignty analysis.
The OSS catalogue for LLMOps: A.10.5 third-party inventory (OSS components with licences and governance).
Anatomy of an LLM request: A.10.2 + A.10.4 in the multi-tenant case.

The roles defined by the standard

ISO/IEC 22989:2022 (AI vocabulary, complementary to 42001) defines six roles. Each organisation has to decide which ones it occupies and document it:

Role	Definition	Main responsibility	Example
AI provider	Organisation that provides the AI system to others	Makes the system available	OpenAI provides GPT-5 via API
AI producer	Organisation that develops the AI system	Design, development, validation	Meta produces Llama 4
AI customer	Organisation that acquires the AI system	Selection, integration, oversight	A consultancy integrating an LLM into its own product
AI partner	Organisation collaborating with another AI role	Shared	A GPU hardware manufacturer
AI subject	Person or group affected by the system	Recipient of the impact	The chatbot’s end user
Relevant authority	Regulator with jurisdiction	External oversight	AEPD, CNMC, EU AI Act authorities

An organisation can occupy several roles at once, which changes the applicable controls. A common pattern in consultancy is: producer + customer + provider towards the end client. The A.10 responsibilities are modulated by the roles.

Example role mapping for the multi-tenant chatbot from the forensic post:

Base model manufacturer (Llama 4): AI producer of the base model.
OSS stack operator (the consultancy): AI producer of the LoRA adapter + AI provider of the chatbot to its clients + AI customer of Meta’s base model.
End client (the insurer): AI customer of the chatbot + AI provider of the customer support service.
The policyholder: AI subject.
AEPD + EU AI Act authority: relevant authority.

Each box in the table generates different obligations. The consultancy, being producer of the adapter, must document A.6 (lifecycle) and A.7 (data) for the adapter. Being provider of the chatbot, it must document A.10.4 (customers). Being customer of the base model, it must document A.10.3 (suppliers) and validate that Meta does its part.

Impact levels and proportionality

42001 does not demand the same rigour of every system. Clause 6.1.2 and control A.5 introduce impact as a modulator. The standard does not define strict categories (unlike the EU AI Act, which does define “prohibited / high risk / limited risk / minimal”), but it recommends using levels according to severity and probability.

Industry practice in 2026 aligns the 42001 levels with the EU AI Act categories:

42001 level	EU AI Act	Examples	Control depth
High	High risk (Annex III)	Credit scoring, HR, health, critical infrastructure	Exhaustive impact assessment, mandatory human oversight, continuous monitoring, adversarial evals, detailed logging, half-yearly management review
Medium	Limited risk	Customer service chatbots that do not automate decisions, productivity assistants	Standard impact assessment, complete guardrails, annual review
Low	Minimal risk	Spam filters, non-personalised content recommendations	Lightweight impact assessment, basic controls

This proportionality is operationally key: implementing 42001 at maximum rigour for a low-risk system is waste; relaxing it on a high-risk one is non-compliance.

The seven minimum AIMS documents

An auditor at Stage 1 (documentation review) asks for between seven and ten documents. The seven essentials:

AI policy (clause 5.2 + A.2.2). 1-2 pages. Approved by management, dated, versioned.
AIMS scope (clause 4.3). List of AI systems in scope, inclusion criteria.
Stakeholder register (clause 4.2 + A.3.4). A maintained list with expectations.
AIMS risk register (clause 6.1). Risks of the management system, not of each model.
Impact assessment procedure (A.5.2) + records of assessments carried out (A.5.3). The procedure plus one or more assessments done.
AI lifecycle procedure (A.6.2), which can literally read “see the six-stage LLMOps pipeline” with references to technical runbooks.
Data management procedure (A.7.2), covering acquisition, quality, provenance, preparation, anonymisation.

Common additional documents:

Responsible use policy (A.9.2) with permitted and non-permitted use types.
AI third-party management procedure (A.10.3, A.10.5) with assessment criteria for AI suppliers.
Internal audit plan + management review agenda (clauses 9.2 + 9.3).

For an organisation with a mature OSS stack, documents 6 and 7 are references to technical artefacts that already exist (pipeline runbooks, DVC configurations, the PII policy in LLM Guard). The real documentary effort sits in documents 1, 2, 3, 4 and 5.

Applied case: the blog’s multi-tenant chatbot → 42001 checklist

We take the system described in the forensic post, a multi-tenant customer support chatbot for insurers on an on-premise OSS stack, and walk it as a 42001 auditor would.

Clause 4 — Context. The AIMS scope includes the chatbot and excludes the internal HR system (a different AI). Stakeholders identified: client insurers, affected policyholders, the AEPD, the EU AI Act authority (once it enters into force on 2 Aug 2026), the supplier Meta (base model), the hardware supplier NVIDIA. → Documented.

Clause 5 — Leadership. AI policy signed by the CEO, in force. Roles assigned: AI lead (CTO), AI risk owner (CISO), data steward (Head of Data), quarterly AI ethics committee. → Documented.

Clause 6 — Planning. AIMS risk register: incomplete documentation, team churn, dependency on a single GPU supplier, EU AI Act regulatory change. AIMS objectives for 2026: 42001 certification before Q4, EU AI Act high-risk compliance before 2 Aug. → Documented.

Clause 7 — Support. Resources: a 4×H100 SXM cluster + the seven stack layers. Competence: 2 MLE + 2 SRE + 1 part-time AI ethics, all with documented training. Communication: AI policy on the intranet + handbook. → Documented.

Clause 8 — Operation. Operating procedures = the six-stage LLMOps pipeline. Impact assessment run before deployment + annual review + review on substantial change (defined as: base model change, major adapter change, expansion to a new tenant). → Documented.

Clause 9 — Evaluation. Monitoring: Langfuse + Tempo + VictoriaMetrics + Grafana. Mandatory dashboard metrics: F1 per guardrail category over real traffic, statistical drift, RAG faithfulness, refusal rate. Quarterly internal audit with written criteria. Half-yearly management review with signed minutes. → Documented.

Clause 10 — Improvement. Incident-driven retrain tickets mapped as nonconformities when severity ≥ HIGH. Documented root cause analysis. Effectiveness verified at the next eval gate. → Documented.

Annex A — By section:

A.2 (Policies): AI policy + responsible use policy. → Documented.
A.3 (Organisation): roles assigned, reporting channel, stakeholder register. → Documented.
A.4 (Resources): seven deployment phases + OSS catalogue + training plan + annual budget. → Documented.
A.5 (Impact): procedure + per-system assessments + fairness metrics applied. → Documented.
A.6 (Lifecycle): LLMOps pipeline + continuous fine-tuning + retrain. → Documented.
A.7 (Data): data versioning + RAG corpus curation + LLM Guard Vault + Presidio. → Documented.
A.8 (Information to parties): OTel tracing + Langfuse + gen_ai.guardrail.* spans + notification to tenants within SLA. → Documented.
A.9 (Use): guardrails + evals + responsible use policy. → Documented.
A.10 (Third parties): OSS vs hyperscalers with lock-in analysis + Meta contract for the base model + contracts with tenants. → Documented.

Result of the walkthrough: certifiable. The typical gaps (A.2.2 written policy, A.3 roles, A.5 impact assessment procedure) are covered as formal documents. The operational clauses (8, 9, 10) lean on the blog’s technical architecture. The distance between “having the architecture” and “having certification” is measured in documentary discipline, not in code.

Cross-mapping with the EU AI Act, NIS2 and ENS

EU AI Act (Reg. 2024/1689) — seven directly aligned articles

EU AI Act article	Topic	Aligned 42001 control	Applies to
Art. 9	Risk management system	A.5 + clause 6	High-risk systems
Art. 10	Data and data governance	A.7 (all)	High-risk systems
Art. 11	Technical documentation	A.6 + A.4.2	High-risk systems
Art. 12	Record-keeping (logs)	A.8.2 + OTel tracing	High-risk systems
Art. 13	Transparency to deployers	A.8.5 + A.10.4	High-risk systems
Art. 14	Human oversight	A.9.2 + documented oversight	High-risk systems
Art. 17	Quality management system	Clauses 4-10	High-risk providers

The main obligations for high-risk systems enter into application on 2 August 2026. Implementing 42001 now builds the management base that deadline demands.

What is missing for EU AI Act compliance that 42001 does not cover:

CE conformity of high-risk systems (declaration of conformity, marking, registration in the EU database).
The specific post-market monitoring required by Art. 72.
Serious incident reporting to authorities within legal deadlines (not only to users).
Transparency obligations towards users for limited-risk systems (Art. 50): chatbots, deepfakes, generated content.
The Art. 5 prohibitions (social scoring, manipulation, real-time biometrics with exceptions).

NIS2 (Dir. 2022/2555) — three pillars with overlap

Asset register (Art. 21.2.f): AI systems within NIS2 scope must be in the asset inventory. → Overlaps with A.4 + clause 4.3 (scope).
Incident notification (Art. 23): significant incidents are notified within 24 h (initial alert) + 72 h (detailed report). → Overlaps with A.3.3 (reporting) + clause 10 (improvement).
Supply chain security (Art. 21.2.d): security assessment of the digital supply chain. → Overlaps with A.10.3 (suppliers).

For essential NIS2 entities that also use AI systems, 42001 covers the AI-specific part that NIS2 requires by inference but does not spell out.

ENS (RD 311/2022)

The Spanish Esquema Nacional de Seguridad already expressly addresses AI in its Annex II (ENS controls). The Basic/Medium/High categories align with the 42001 impact levels. The ENS controls on traceability (op.exp.8), activity logging (op.exp.10) and incident management (op.exp.7) are covered by the same technical artefacts as A.8 and A.5 of 42001. An organisation certified at ENS High category with AI systems is a reasonable effort away from adding 42001.

The five usual certification traps

Trap 1 — Confusing 42001 with EU AI Act compliance. Passing the 42001 audit does not imply conformity with the European Regulation. They are distinct universes with a 60-70% overlap. The trap surfaces when the client asks for CE marking of the high-risk system and the organisation produces only the 42001 certificate.

Trap 2 — Over-documenting. Two-hundred-page manuals with procedures copied from templates, with no measurables and no evidence of application. The auditor asks for the last execution of the procedure; with no records, the procedures are ornament. The practical rule: prefer short documents referencing live technical artefacts to long self-contained ones.

Trap 3 — Under-measuring. Defining AIMS objectives with no operational metrics. “Improve model quality” is a null objective; “F1 per guardrail category ≥ 0.85 over real traffic, measured weekly, reviewed quarterly in the management review” is an auditable objective. The blog has insisted on this in every post on evals, guardrails and retrain.

Trap 4 — Ignoring A.5 until audit day. The impact assessment is the most underrated control and the first one the auditor asks for. Without per-system assessments carried out before deployment, there is no way to demonstrate A.5. The trap surfaces when there is no longer time to produce credible retrospective assessments.

Trap 5 — Assuming 27001 covers the AI part. Organisations that already have 27001 sometimes think “we have half of it done”. That is true for Annex SL (structure) and for A.5/A.6/A.7 of 27001 (not of 42001) as far as infosec goes. It is false for A.5 of 42001 (impact assessment), A.7 of 42001 (AI-specific data quality), A.9 (responsible use) and A.10.4 (AI customers). You have to add, not assume.

What we have not covered (upcoming posts)

Concrete templates of the seven mandatory documents, with drafting examples and metrics. Material for a post along the lines of “The AIMS manual in 7 documents” with a reference frame.
Detailed article-by-article mapping to the EU AI Act with the checklist of technical evidence derivable from the blog’s OSS stack. Especially Arts 11 (technical documentation), 14 (human oversight) and 72 (post-market monitoring).
The combined ENS High category + 42001 case: which ENS controls are covered by which AIMS artefacts, avoiding duplication.
NIST AI RMF 1.0 vs 42001 comparison: many international clients ask for both. How the same artefacts are recycled to satisfy the two frameworks.
42001 for LLM agents and MCP: the new dimensions that emerge when the AI system is agentic (excessive agency, tool use, graduated autonomy). The guardrails post introduced line 3 (tool GR); 42001 has open gaps in this terrain and SC 42 is working on addenda.

References

ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system. ISO. https://www.iso.org/standard/81230.html.
ISO/IEC 22989:2022 — Information technology — Artificial intelligence — Artificial intelligence concepts and terminology. Define los roles AI provider/producer/customer/partner/subject.
ISO/IEC 23894:2023 — Information technology — Artificial intelligence — Guidance on risk management. Insumo de A.5.
ISO/IEC 38507:2022 — Governance implications of the use of AI by organizations. Complemento de gobierno.
ISO/IEC 5259 — Data quality for analytics and machine learning (familia). Insumo de A.7.
EU AI Act (Regulation 2024/1689) — texto consolidado en EUR-Lex. Entrada en vigor de obligaciones de alto riesgo: 2 ago 2026.
NIS2 (Directive 2022/2555) — texto consolidado en EUR-Lex.
ENS — Real Decreto 311/2022 — Esquema Nacional de Seguridad, BOE-A-2022-7191.
NIST AI RMF 1.0 (2023) — https://www.nist.gov/itl/ai-risk-management-framework.
EUR-Lex EU AI Act consolidated text — https://eur-lex.europa.eu/eli/reg/2024/1689.
A-LIGN / BSI / Schellman — blogs sobre experiencia de auditoría 42001 con casos reales 2024-2025.

RAG corpus curation: the active librarian who decides what goes in, what goes out and what gets signed

Mon, 25 May 2026 11:00:00 +0200

This is the curation layer inside stage 1 (Data) of the six-stage LLMOps pipeline. It complements the other Data posts: dataset versioning for the four versionable artefacts, PostgreSQL + Qdrant ingestion in microservices for the outbox + CDC pattern, and RAG over Kafka and a datalake for streaming transport. This post is not about moving data: it is about what to do with data before you let a model read it.

TL;DR

A RAG system serving mediocre answers is rarely the retriever’s fault, nor the model’s. The root cause is usually the corpus: three nearly identical versions of the same PDF that make top-k return the same thing three times, an old manual nobody removed that contradicts the current one, a free-text field with customer numbers that the model quotes verbatim, a scanned PDF with dirty OCR that the chunker cut in the middle of a sentence. None of that is fixed by changing the model, the embedder, the reranker or the prompt. It is fixed by curating the corpus. This post takes apart the five operational layers of curation (schema-validated ingest, three-level deduplication, PII anonymisation measured with precision/recall, anti-contamination against the golden eval set, chunk→trace lineage), the minimum maths needed to avoid fooling yourself, the 2026 stack (Presidio, Unstructured, Argilla, LangChain text splitters, OpenLineage, Marquez, Great Expectations), the seven traps that reduce the stage to theatre, and the on-premise hardware to sustain all of it without sending anything sensitive to external APIs.

The analogy: the active librarian

A serious librarian does not accept books by the kilo. When someone offers a new volume:

They look at the spine, the ISBN and the stamp: is it legible? Is it catalogued correctly? Does it belong to a recognised collection? Without valid metadata, it does not get in. This is the schema check.
They check whether they already hold a copy: is it exactly the same book? Is it a later edition of the same one? Is it a translated version of something already on the shelves? If so, they decide explicitly what to do (replace, archive the old one, withdraw both from lending). This is dedup at its three levels.
They flag what is restricted: if the book contains identifiable personal data, some pages cannot be lent as they are, so they must be redacted, anonymised or moved to the reserved section. This is PII anonymisation.
They verify it is not this year’s final exam book: if it is, it stays out of the public collection until the syllabus changes, because if students can consult it the exam stops measuring what it is meant to measure. This is anti-contamination against the golden eval set.
They write it into the register: this book, this edition, this provenance, this date, this person who approved the entry. This is lineage.

If the book passes all five, it joins the collection. If it fails any of them, it goes to an auditable quarantine shelf with the reason for rejection. The difference between a good collection and a mediocre one is not size: it is how much discipline you apply across the five layers, every day, to every new book that arrives.

A RAG corpus is exactly that. The only differences are scale (thousands or millions of documents per month) and the fact that the “readers” are LLMs that cannot tell a duplicate from a reinforced truth, nor a PII value from a synthetic example, nor a contaminated fragment from an authentic one.

The four data artefacts and where the RAG corpus fits

Before going down to the five layers it is worth being clear about which corpus we are curating. The Data stage of the pipeline manages four distinct artefacts, each with its own discipline. The data versioning post lists them; here I reorder them from a curation perspective:

Artefact	Who consumes it	Dominant curation
Training dataset	Tune (fine-tuning the model or an adapter)	aggressive dedup + quality filters + label balancing
RAG corpus	Deploy (retrieval at request time)	the 5 layers in this post
Golden eval set	Eval (promotion gates)	strict hold-out + stratification + maintenance driven by incidents
Retrain enriched dataset	Retrain (closing the loop)	production feedback + human triage

The RAG corpus is the most volatile of the four and the one most exposed to the end user: every answer the system serves literally contains fragments of it. A duplicate in the training dataset degrades learning but ends up buried in the weights; a duplicate in the RAG corpus shows up in today’s answer and tomorrow’s. That justifies the extra discipline that follows.

Layer 1 — Schema-validated ingest

Every piece entering the corpus has to arrive with structured metadata validated against a schema. This is not bureaucracy: it is the only way to make the following layers (dedup, PII, lineage) work without friction.

The canonical pattern is to define a schema in JSON Schema or Pydantic that every document must satisfy:

class CorpusDocument(BaseModel):
 source_system: str # e.g. "confluence", "salesforce", "manual_pdf"
 source_id: str # unique ID in the source system
 version: str # document version (semver or date)
 language: str # ISO 639-1: "es", "en"
 title: str
 body: str
 captured_at: datetime
 captured_by: str # pipeline or human
 sensitivity: Literal["public", "internal", "restricted"]
 schema_version: str # version of the schema itself, not of the document

Any piece that fails this contract is rejected at ingest and never reaches the following layers. Validation is done with Great Expectations (declarative suites), Pandera (more pythonic, integrates with pandas) or Soda (oriented towards continuous data quality). The choice is a matter of style; what matters is:

Validation suites live in code and are versioned with the pipeline, not in a notebook off to one side.
Rejection generates an auditable event (quarantine) with the specific schema failure as its reason, not a log line lost in stdout.
The schema itself is versioned. When it changes, previous documents are reprocessed or explicit backward compatibility is maintained.

The RAG over Kafka post covers the Schema Registry pattern (Confluent Schema Registry or Apicurio) that materialises this in streaming: every message on the topic is validated against the registered schema before propagating downstream. For batch or pull, Great Expectations is the equivalent.

Common trap: leaving the body field free with no further validation. You have to tighten it up with minimum/maximum length (a PDF that yields 12 characters after extraction is almost certainly broken), valid encoding (UTF-8 with no control characters), proportion of alphanumeric characters (dirty OCR returns a soup of symbols). These are simple rules that filter 80% of the noise with no need for AI.

Layer 2 — Three-level deduplication

The most expensive and most silent mistake in a RAG corpus is the duplicate. A document that appears three times in the corpus makes retrieval’s top-k return it three times, wasting two slots and reinforcing a single source. The LLM reads it as if three independent sources agreed, when in fact it is the same thing repeated.

Deduplication is done at three levels, in this order by cost:

Level A — Exact dedup (SHA-256 hash)

Compute the hash of the normalised content (trim, lower-case where applicable, remove redundant whitespace) and compare it against an index of already ingested hashes. On a match, discard or replace. Cost: \(O(1)\) per document. It catches literal duplicates (the same PDF uploaded twice, two byte-for-byte copies of the same HTML).

Level B — Near-duplicate (MinHash + LSH)

Nearly identical documents with minor differences (a different header, an updated date, one version in Castilian and another in Galician with minimal changes). The canonical algorithm is MinHash with Locality-Sensitive Hashing (LSH), which approximates Jaccard similarity over shingles of k tokens. For n documents, comparing all against all is \(O(n^2)\), which is unworkable for large corpora. LSH cuts the cost to \(O(n)\) most-likely buckets.

A typical threshold is Jaccard ≥ 0.80 over shingles of 5 tokens. The standard libraries are datasketch (Python, MIT) or dedup (Python, MIT). A numerical example: for 1 M short documents (300 tokens each), datasketch.MinHashLSH with 128 permutations and threshold 0.8 takes about 2 GB of RAM and processes the full corpus in around 30 minutes on a modern CPU. The fraction of duplicates found in a real enterprise corpus is usually between 5% and 25%; removing them cuts storage and improves retrieval quality at the same time.

Level C — Semantic dedup (cosine over embeddings)

Documents saying the same thing in different words, such as paraphrases, translations or rewritten versions, are not caught by MinHash. This is where semantic similarity comes in: compute the embedding of each document and compare the cosine between pairs.

The problem is quadratic cost: for n documents, computing every pair is \(O(n^2)\). For n = 1 M and 768-dimensional embeddings (a typical model such as BAAI/bge-base-en-v1.5), that is 5×10^11 dot products, which is unworkable. The solution is the same idea as LSH but over dense vectors: HNSW (Hierarchical Navigable Small World) or IVF (Inverted File) to build an approximate search index. For each new document you run a k-NN query against the index and examine only the k nearest neighbours.

A sensible threshold for treating something as a semantic duplicate: cosine ≥ 0.95. Below 0.95 the documents are related but distinct; above it, they are almost always the same information rewritten. The exact threshold is calibrated by observing precision/recall over a human-annotated sample; 100 pairs confirmed by a reviewer is reasonable for setting it.

A numerical example: with qdrant or pgvector as the HNSW index and k=10 neighbours per query, deduplicating 1 M documents against the existing corpus takes on the order of 2-4 hours on an RTX 4090 (including embedding computation). If the embedder is self-hosted with vLLM, the cost per token is negligible against compute time.

Policy on what to do with a duplicate

Detection is not enough, you have to decide. Three common policies, in order of complexity:

Drop: discard the more recent one, keep the older one. Simple, with no extra lineage.
Replace: discard the old one, index the new one. More volatility but it reflects the update.
Merge with provenance: mark the new one as a “shadow” of the old one, keep both in lineage but index only one. Better for regulated audit.

The policy has to be explicit and applied uniformly, not an ad-hoc decision per document.

Layer 3 — PII anonymisation with measured precision/recall

This layer is the one that most easily turns into theatre. The typical mistake: install Presidio, run it over the corpus, assume the output is clean. Without measuring the detector’s precision and recall against an annotated golden set, you know nothing.

A PII detector can fail in two ways:

False negative (low recall): it does not detect a national ID written as “12345678-A” because your model was trained on the 12345678A format without a hyphen. The RAG serves personal data unredacted.
False positive (low precision): it redacts a number from a configuration manual thinking it is a phone number. The RAG loses useful information.

Both are problems; regulation (GDPR, ENS, NIS2) penalises the first, and user experience degrades with the second. The acceptable ratio depends on the domain. In medical data, practically zero false negatives is non-negotiable; in internal technical documentation you can tolerate more recall in exchange for less precision.

The standard metric is F1 over an annotated golden set:

\[ \text{precision} = \frac{TP}{TP + FP}, \quad \text{recall} = \frac{TP}{TP + FN}, \quad F_1 = 2 \cdot \frac{\text{precision} \cdot \text{recall}}{\text{precision} + \text{recall}} \]

To build the PII golden set, annotate around 200 documents by hand with every entity marked (national ID, IBAN, email, phone, address, personal name). Then run the detector and compute the metrics per category, not just aggregated, because a global F1 of 0.90 can hide a recall of 0.55 on IBANs.

2026 stack for this layer:

Microsoft Presidio (MIT, Microsoft): the most complete OSS option. Configurable detectors, recognises around 50 entities by default, extensible with your own regex patterns or with fine-tuned NER models.
spaCy NER (MIT, Explosion AI): a base for custom detectors; useful when Presidio does not cover a domain entity.
Llama Guard 4 (Llama Community License, Meta): a safety classifier that also detects PII in one pass, an option when you already have a GPU for inference and prefer a single pass.
DataFog (Apache 2.0): a more recent alternative, specialised in streaming pipelines.

Recommended hybrid pattern: Presidio for rule-based + regex detection (fast, deterministic) → Llama Guard as a second pass over what Presidio did not flag (an ensemble that raises recall without killing throughput). This is measured and reported as aggregate and per-category F1 on every release of the detector.

Common fallacy: trusting that a detector with F1 0.95 “is very good”. If you have 1 M documents and each contains on average 1 PII entity, F1 0.95 means 50,000 mishandled entities (between false positives and negatives). With sensitive data you have to design so that false negatives go to human quarantine, not to the public corpus.

Layer 4 — Anti-contamination against the golden eval set

If the RAG corpus contains fragments of the golden eval set, Eval metrics measure memorisation. The model returns the exact answer because it has it literally in its context, not because it generalised anything. Deploy then promotes models that shine in the exam and fail in production.

This layer is the easiest to implement and the easiest to forget:

The golden eval set has its own versioned hash.
Before indexing any new document into the RAG corpus, run a token-by-token overlap check (or by shingles, similar to MinHash) against the golden set.
If overlap exceeds a threshold (typically ≥ 30% of 5-token n-grams), the document is not indexed. It stays in quarantine flagged as “contamination risk vs golden_v12”.
A human reviews the rejections. Sometimes they are false positives (a short quotation, a boilerplate sentence). Sometimes they are real contamination that a supplier introduced without realising.

The deeper reason: the RAG corpus and the golden set are enemies by design. The golden set measures how well the system generalises to questions it has not seen. If those questions are in the RAG, the system “sees” them on every query. The metric stops measuring generalisation.

This check is computationally trivial, a hash join over n-grams. The difficulty is keeping it up: every time the golden set changes (monthly or quarterly), the full corpus has to be re-validated against the new golden set. Without that discipline, contamination sneaks in through the back door when someone updates the golden set with real cases that the RAG was already serving.

Layer 5 — End-to-end lineage: from document to trace

The last layer is the one that closes the auditable chain. Every chunk indexed in the vector store carries metadata that lets you answer the forensic question:

“The system generated this answer on 14 March at 16:23. Which exact document did the quoted fragment come from? When did that document enter the corpus? Which version of the embedder processed it? Who approved its ingest?”

Without lineage, that question is unanswerable. With lineage done properly, it is four queries.

The canonical pattern:

Every indexed chunk carries in its metadata: source_system, source_id, document_version, chunk_index, embedder_version, dataset_hash, ingested_at, ingested_by, schema_version.
Every RAG answer in production emits a trace span that includes the retrieved chunk_id values.
The central tracing system (Langfuse, Phoenix or OpenLLMetry) joins chunk_id → chunk metadata → document metadata → corpus dataset_hash → embedder version → and so on.

The tools that standardise this glue are OpenLineage (Apache 2.0, LF AI & Data) and Marquez (Apache 2.0, its server implementation). They define a lineage event schema that interoperates between systems; an ingest job emits an event “produced corpus_v12.3 from source X with embedder bge-base-v1.5”; a retrieval job emits “consumed corpus_v12.3 with query Q produced response R”. The graph is reconstructed automatically.

This layer is the only way to pass real audits under regulations such as the EU AI Act, GDPR or ENS, where traceability of which data went into which answer is a requirement, not an option. Without it, the answer “we do not know which document this came from” is not acceptable, and it is the default answer if lineage is not built from day one.

The maths that matter

Beyond dedup thresholds and PII F1 scores, there are three mathematical pieces that any serious team ends up using.

Chunk size vs retrieval quality. Chunk size affects retrieval quality in a non-monotonic way: chunks that are too small fragment ideas (retrieval returns a piece without context), and chunks that are too large dilute the signal (the embedding mixes several topics and similarity drops). The empirical sweet spot for technical text in 2026 is between 256 and 768 tokens per chunk, with 15-25% overlap between contiguous chunks to preserve continuity.

Numerically, for a corpus of 1 M documents with a mean length of 2,000 tokens, chunking at 512 tokens with overlap 100 gives \(\frac{2000}{512 - 100} \approx 5\) chunks per document, a total of roughly 5 M indexed chunks. With 768-dimensional embeddings in float32, that occupies \(5 \cdot 10^6 \cdot 768 \cdot 4 \approx 15\) GB of vector memory, manageable in any modern vector store.

PII golden set coverage. To know whether the annotated PII golden set is representative enough, compute the proportion of categories covered: if your golden set of 200 documents has 5 IBAN examples and production has 12,000 IBANs per day, the measured F1 on IBANs is statistical noise. Rule of thumb: a minimum of 30 examples per category for per-category metrics to mean anything.

Cost of re-embedding when rotating the model. Changing the embedder invalidates the whole index. For a corpus of 5 M chunks with a model such as BAAI/bge-base-en-v1.5 (768 dim, around 110 M parameters) served in vLLM on 1× H100, throughput is on the order of 8,000-15,000 chunks/second. Full re-embedding: about 5-10 minutes. For a larger embedder (bge-large, 1024 dim, around 335 M parameters), a factor of 3× worse, around 15-30 minutes. The bottleneck is usually vector store I/O, not GPU compute. The dual-index pattern, keeping the old index serving while the new one is built and doing an atomic swap at the end, avoids downtime and allows rollback.

Applied to typical on-premise hardware

For an on-premise deployment that keeps all curation in-house without sending data to external APIs:

RTX 4090 (24 GB): covers layer 1 (the schema check with Great Expectations is CPU-bound), layer 2 levels A and B (hash + MinHash are CPU-bound), and layer 2 level C semantic dedup with a bge-base-class embedder (8-15k chunks/s, enough for corpora of up to 5-10 M chunks in hours). For Presidio in NER mode (layer 3) it runs comfortably. It is the sensible GPU for the whole curation pipeline on a mid-size corpus.
Generic 4×H100 SXM configuration (320 GB total, NVLink): only needed if the corpus exceeds around 50 M chunks or if you want frequent re-embeddings with large models (bge-large, e5-mistral). In practice, two GPUs serve the embedder at TP=2 with throughput above 50k chunks/s, and the other two go to the PII judge (Llama Guard 4) or to serving the main inference model. Capacity for corpora of hundreds of millions of chunks.

The stubborn arithmetic: with a 4090, corpus curation is an overnight job; with 4×H100, it is minutes. The decision depends on corpus size and on how often you rotate the embedder or the PII rules.

The seven traps that kill this stage

Trap 1 — No validated schema at ingest. Malformed documents reach the chunker, the chunker slices them into nonsense, garbage embeddings enter the index. The RAG answer quotes incoherent text and nobody knows why.

Trap 2 — Dedup only at exact hash level. The corpus fills up with paraphrases and translations of the same document. Retrieval’s top-k returns the same source 3 times. The LLM reads it as three confirmations.

Trap 3 — PII detector without precision/recall measurement. Everyone assumes Presidio “works”. IBANs in non-standard formats slip through. The RAG serves personal data.

Trap 4 — Golden eval set contaminated with the corpus. Eval metrics measure memorisation. Promotions approve models that fail in real production.

Trap 5 — No lineage down to the chunk. The question “where did this quotation come from?” has no answer. Regulatory audit fails. Incidents cannot be investigated.

Trap 6 — Maintenance as a one-off event. The corpus is curated once when the system is initialised, and afterwards everyone assumes it is fine. Six months later the documents are out of date, the new PII rules are not applied retrospectively, dedup has not been re-run after adding new sources. The corpus degrades silently.

Trap 7 — Quarantine without human review. Rejected documents go into a table nobody looks at. False positives pile up, real contamination cases go uninvestigated, the team’s trust in curation erodes and the pressure to “relax the thresholds” begins.

All seven are operational, not technical. Corpus curation does not break because of an algorithm bug: it breaks because discipline slackens. It is the exact equivalent of the kind of degradation that kills Eval suites, and in both cases the symptom is the same: the metrics improve or hold steady while the real experience gets worse.

What we have not covered (upcoming posts)

Vector store versioning proper: an embedding index is not versioned like a raw dataset because it depends on the embedding model. Changing the embedder rewrites the whole index. It is a different animal with its own patterns (index branching, selective reembedding, recall-aware ANN parameters).
Streaming corpus updates with CDC: when the corpus has to be updated in near-real time from an OLTP system. The Postgres + Qdrant ingestion post covers the mechanics; still pending is the pattern for selectively invalidating chunks that depend on deleted rows.
Multi-tenant corpus isolation: how to set up a shared corpus versus one with per-tenant namespaces, with ACLs over individual chunks. Especially relevant for multi-customer RAG under data sovereignty constraints.
Federated corpus: corpora distributed across silos that the system queries without centralising the content. An emerging pattern for companies with several sites and cross-border restrictions.
Reranking-aware curation: how curation discipline changes when there is a reranker (Cohere Rerank, ColBERTv2, BGE-Reranker) reordering the top-k after retrieval. Some duplicates you would tolerate without a reranker become intolerable when the reranker pushes them up the ranking.

References

Presidio: https://microsoft.github.io/presidio/ — official docs, list of supported entities, guide to extending it with custom NER.
OpenLineage: https://openlineage.io/ — event schema spec and libraries per language.
Marquez: https://marquezproject.ai/ — server implementation of OpenLineage.
datasketch (MinHash + LSH): https://ekzhu.com/datasketch/ — the reference Python library for near-duplicate deduplication at scale.
Great Expectations: https://docs.greatexpectations.io/ — declarative data quality suites.
Unstructured: https://docs.unstructured.io/ — parsing and normalisation of heterogeneous documents (PDF, HTML, DOCX, eml) before chunking.
Argilla: https://docs.argilla.io/ — human annotation UI for building the PII golden set and other calibration sets.
Llama Guard 4: Meta technical paper, multimodal safety classifier — useful as a second PII detection layer.
GDPR, EU AI Act, ENS, NIS2 — the regulatory frameworks whose compliance depends, in practice, on the discipline of layers 3 (PII) and 5 (lineage). The CEN/CENELEC technical standards for conformity assessment of GenAI systems under the EU AI Act are still pending final publication.

Data versioning for LLMOps: DVC, lakeFS and the reproducible golden dataset challenge

Fri, 22 May 2026 11:00:00 +0200

TL;DR

The Data stage of the six-stage LLMOps pipeline has a silent link on which everything else depends: versioning the datasets with the same discipline the code is versioned with. It is not optional. An LLM system in production consumes at least four distinct types of dataset (training/fine-tuning, RAG corpus, golden eval set, enriched dataset from the Retrain loop) and each has its own demands. Git solves code but fails on data for two technical reasons (size and a useless binary diff) and one operational one (it does not propagate lineage as far as the weights bucket of the trained model). The two dominant OSS tools, DVC and lakeFS, unified in November 2025 under a single organisation with a roadmap oriented towards LLM training and RAG datalakes; they are still complementary projects (file-level vs whole-bucket branching) but now under common governance. The productive pattern the market has consolidated on: identify each artefact with an immutable (dataset_id, version), propagate the pair as far as the experiment tracking (MLflow / W&B), version the dataset’s schema too (not just its content), apply a strict holdout to the golden eval set so you are not measuring memorisation, and maintain bidirectional traceability dataset_version ↔ model_version ↔ deployment ↔ trace_id. Without this, the promise of “we can audit which model answered what” collapses at the first serious incident.

You are here: Data (with a cross-cutting effect on Tune, Eval and Retrain)

This post goes into the detail of the versioning link inside stage 1 · Data. Versioning belongs operationally to Data, but the artefacts it produces travel to Tune (training set), Eval (golden set) and Retrain (enriched dataset). That is why the diagram marks Data as active and shows a cross-cutting band indicating end-to-end lineage.

The master analogy: batch traceability in a serious factory

A serious pharmaceutical factory does not produce without batch traceability. Every box of pills carries a printed batch number; that batch is associated with manufacturing dates, with the specific batches of each raw material used, with the quality tests it passed, and with the technicians who signed off each step. If a patient reports an adverse effect, the factory can rewind within hours: this package → this batch → these raw materials → this shift → this production line → this quality control result. Without that chain, the incident is a permanent mystery.

A serious LLM system works the same way. The “package” is the answer a user saw in production. The “batch” is the combination of model, adapter, prompt, context and configuration that generated it. And the “raw materials” are the datasets: the training set the base model was trained on, the adapter’s fine-tuning dataset, the RAG corpus that feeds retrieval, the golden eval set that validates promotion. If a customer says “what data was the model trained on that answered X to my question Y on 14 March?”, without batch traceability the answer is “we do not know”. And that, with a customer under compliance pressure, kills the contract.

Git versions the recipe (the code). Data versioning versions the ingredients. Without both, there is no auditable factory.

The four artefacts worth versioning (with differentiated demands)

Not all datasets are versioned the same way or at the same frequency. The typical LLM system in production handles four artefacts worth governing separately.

Artefact	What it is	Typical size	New-version frequency	Who consumes it
Training / fine-tuning dataset	Input/output pairs (or conversations) that train the adapter or the model.	10⁴ – 10⁷ examples · 1 – 100 GB	Per Tune experiment	Trainer (Axolotl, TRL, Unsloth)
RAG corpus	Indexed documents that feed retrieval.	10⁵ – 10⁹ chunks · 10 GB – 10 TB	Almost continuous (streaming ingest)	Indexer + vector store
Golden eval set	Curated examples with an expected answer to measure quality.	10² – 10⁴ examples · MB	Per product release	Eval gates in CI
Enriched retrain dataset	Cases where the system failed + the human correction.	Hundreds to thousands per quarter	Per retrain cycle	Next Tune

All four have common requirements (immutable identity, lineage, schema) and relevant differences:

The training set tends to be large, stable per experiment, and the cost of an error is a lost experiment (expensive but bounded).
The RAG corpus is enormous, in continuous change, and its versioning is handled through periodic snapshots of the index (not of the raw text). Usually lakeFS or bucket branches; DVC is not the best fit.
The golden eval set is small but critical: errors here contaminate the whole promotion chain. Here the rigidity of the versioning matters more than anywhere else.
The enriched retrain dataset is incremental by nature: each Retrain cycle contributes a delta over the previous one. The new version does not overwrite; it inherits and adds.

Confusing them, treating the RAG corpus as if it were the training set, or the golden eval as if it were just another dataset, is the source of half the operational problems in data versioning.

Why Git is not enough

The obvious question: if Git already solves code, why does it not solve data too? Three reasons, two technical and one operational.

Reason 1: size. A Git repository with a 50 GB dataset becomes unmanageable. git clone pulls the entire history; git status walks every file; the pack file in .git/objects bloats to up to twice the size of the dataset. Git LFS solves the first part (the binary leaves the pack) but introduces its own complexity without addressing the other two reasons.

Reason 2: useless binary diff. Git assumes text diffs are useful. When a column changes in an 8 GB parquet, the diff is opaque, because the file is binary, compressed and columnar. You cannot do code review on a dataset change the way you do on a function change. You need a semantic diff: how many rows changed, which columns changed, which distribution moved. No native Git gives you that.

Reason 3: lineage that crosses repository boundaries. This is the most important and the most subtle. The training dataset lives in a bucket. The trainer code lives in a Git repo. The trained model is published to a model registry. Inference in production generates traces in an observability system. Connecting dataset_v3 → adapter_v7 → deployment_d2 → trace t_x9 requires propagating identifiers across four different systems, not inside one repo. Git has no opinion on this.

Data versioning tools (DVC, lakeFS, Pachyderm, Quilt) exist because they solve the three problems at once: they hang the data outside the Git repo, they offer some form of semantic diff, and they expose stable identities that can be propagated towards experiment tracking and the model registry.

DVC vs lakeFS before the unification

Until November 2025, the two dominant OSS tools coexisted as complementary approaches.

Axis	DVC	lakeFS
Mental model	“Git for data”	“Branching for the data lake”
Granularity	Individual file	Whole bucket (with per-branch namespacing)
Storage	Remote-agnostic (S3, GCS, Azure, MinIO, SSH)	S3-compatible (S3, MinIO, Ceph)
Workflow	`dvc add` + `dvc push` + `dvc.yaml` pipelines	`lakectl commit` + branches/merges over the bucket
Diff	File hash + external metadata	Object-level diff + commit log
Strong cases	Discrete training datasets, model files, reproducible pipelines	Large RAG corpora, branching a shared data lake, parallel experiments without duplicating data
Git integration	Deep (the `.dvc` files are committed to Git)	Tangential (lakeFS lives alongside)
Who operates it	MLE team	Data engineering team

In practice, many teams used them at the same time: DVC for the discrete datasets that fed an experiment (they fit in a Git repo thanks to the indirection of the .dvc pointers), and lakeFS for the large RAG corpus bucket they wanted to branch without duplicating terabytes.

What changed with the November 2025 acquisition

lakeFS acquired DVC in November 2025. The operational consequence as of May 2026 is modest but relevant:

There is (as yet) no technical merger of the projects. DVC is still DVC and lakeFS is still lakeFS. The current CLIs, formats and workflows have not changed.
An explicit combined roadmap towards LLM training and RAG datalakes. The merged organisation has stated specific priorities: consistent branching between the dataset and the trained model, native integrations with MLflow / W&B / Langfuse, support for the typical LLM formats (jsonl, parquet with embedded tokenisation), and branch-aware vector indexing.
Convergence expected in 2026-2027. The market anticipates a single registry with two operating modes (file-level + bucket-branching) under a unified CLI. As of today, teams still combine both.

The practical reading for 2026: adopt DVC for discrete training/eval datasets and lakeFS for the RAG corpus, but design the lineage so that a future unified registry can absorb both without re-versioning everything. Concretely: use stable identifiers (dataset_id, version, commit_hash) that can be propagated regardless of the tool.

The operational pattern: four-hop lineage

Once you accept that datasets have to be versioned, the question is not “which tool” but “which chain of identifiers connects production to the source data”. The pattern the market has consolidated on has four hops:

(dataset_id, dataset_version)
│ versioned in DVC or lakeFS
▼
(model_id, model_version)
│ registered in MLflow / W&B with the dataset as input
▼
(deployment_id, prompt_version)
│ registered in the model registry + prompt registry
▼
(trace_id)
│ emitted by the inference engine with OTel
▼
answer visible to the user

Each arrow is a metadata write that crosses the boundary between two systems. If a single arrow is missing, the lineage breaks and the promise of auditability evaporates.

A concrete example of the flow, using DVC + MLflow:

# Data stage: version the dataset
dvc add data/finetune_v3.jsonl
git add data/finetune_v3.jsonl.dvc data/.gitignore
git commit -m "data: finetune dataset v3"
dvc push # uploads the binary to the remote (MinIO/S3)

# Tune stage: train while recording lineage
mlflow run train.py \
 -P dataset_id=finetune \
 -P dataset_version=v3 \
 -P dataset_hash=$(dvc get-url data/finetune_v3.jsonl | sha256sum)
# The run records: input dataset + model output

# Eval stage: validate while recording lineage
mlflow run eval.py \
 -P model_id=adapter_customer_v7 \
 -P golden_set_id=customer_support \
 -P golden_set_version=v12

# Deploy stage: the deployment inherits the dataset + golden ids
# Every trace in Observe carries model_version + prompt_version
# which rewind back to dataset_version

The equivalent version with lakeFS over the RAG corpus:

# Branch for the new corpus embeddings
lakectl branch create lakefs://corpus/embed-2026q2 --source main

# Index the corpus on that branch
python index_corpus.py --branch embed-2026q2

# Validate before merging to main
python eval_retrieval.py --branch embed-2026q2 \
 --metric recall@10 --threshold 0.78

# If it passes, merge (changes the corpus that serves production)
lakectl commit lakefs://corpus/embed-2026q2 -m "embed: corpus 2026q2"
lakectl merge lakefs://corpus/embed-2026q2 lakefs://corpus/main

The virtue of the second flow: while the new corpus is being validated, the production system keeps serving from main with no interference. The parallel branch works as a real staging area over the whole bucket.

Schema contracts: data versioning without them is an illusion

Versioning a dataset’s content without versioning its schema is a frequent mistake. The problem: a versioned dataset with an implicit schema still breaks silently when a producer (the ingestion team, the annotation team, an ad-hoc script) changes a field.

A concrete case: a customer support golden eval set, 1000 examples, an expected_output field originally string. Someone decides they need to capture several valid answers and changes the field to list[string]. The eval loader accepts both formats by chance (Python is lax) but the downstream LLM judge receives a different object. The eval keeps passing but now it measures something else.

Productive pattern: the dataset is versioned with DVC/lakeFS and its schema is versioned with a Schema Registry (Confluent or Apicurio) or, in less mature systems, with a JSON Schema embedded alongside the dataset. CI blocks any PR that breaks the contract without a version bump.

A minimal schema for a golden eval entry (illustrative):

$schema: https://json-schema.org/draft/2020-12/schema
$id: https://example.org/schemas/golden_eval_entry/v3.json
type: object
required: [example_id, input, expected_outputs, rubric, segment]
properties:
 example_id: {type: string, format: uuid}
 input:
 type: object
 required: [user_query, retrieved_context]
 properties:
 user_query: {type: string}
 retrieved_context: {type: array, items: {type: string}}
 expected_outputs:
 type: array
 minItems: 1
 items: {type: string}
 rubric:
 type: object
 required: [must_include, must_not_include, format]
 properties:
 must_include: {type: array, items: {type: string}}
 must_not_include: {type: array, items: {type: string}}
 format: {enum: [text, json, markdown]}
 segment: {type: string}
 difficulty: {enum: [easy, medium, hard]}
 added_at: {type: string, format: date-time}
 curated_by: {type: string}

Operational rules:

Explicit forward/backward compatibility: adding an optional field is backward-compatible; removing a required one is breaking. The policy is enforced with a compatibility check in CI.
The schema version embedded in every row of the dataset (a _schema_version field). The loader validates that the version matches what the consuming code expects.
The schema registry as the single source of truth, not as an optional copy of the JSON Schema in four repos.

Without this level of discipline, “we have data versioning” means “we store the bytes but we do not control what they mean”.

Golden eval set: the most critical version

Of the four artefacts, the golden eval set is the one that demands the most rigour. A failure here contaminates the whole promotion chain: if the eval lies, the gates approve models that should not pass.

Three extra disciplines over the golden set:

Annotation with measured quality. Each example is labelled by a human, and a percentage (10-20 %) is annotated by two independent people. The inter-annotator agreement (Cohen’s kappa or pairwise F1) is measured and published; a golden set with kappa < 0.7 is measuring human noise, not model behaviour. Argilla and Label Studio provide the mechanics; what matters is the discipline, not the tool.

A strict holdout against contamination. The golden set must never enter the training set. The concrete mechanism: hash each golden set input (sha256 normalised by lowercasing + stripping trivial punctuation) → check in CI against all the training set hashes. If there is an intersection, CI blocks until it is resolved. Without this check, the model passes the eval by memorisation, not by capability. The consequence in production is disastrous: the “validated” model fails on cases analogous to the golden set that it had not memorised.

Additive versioning, never destructive. When the golden set grows (each retrain cycle adds cases), golden_v3 = golden_v2 ∪ new_examples. Never golden_v3 = a different new set. Only that way can you compare two models trained months apart over the same base + the new delta. If you rewrite the golden set, you cannot tell whether the March model was worse than the May one or whether you were simply measuring different things.

Summary table of the discipline per artefact:

Practice	Training set	RAG corpus	Golden eval set	Enriched retrain
Immutable versioning	Yes	Yes (snapshots)	Yes, critical	Yes
Schema with a contract	Yes	Recommended	Yes, critical	Yes
Double annotation	No	Not applicable	Yes (10-20 %)	Yes (10-20 %)
Holdout vs other datasets	N/A	N/A	Yes, hash check	Yes (vs golden)
Drift check vs previous version	Recommended	Yes	Recommended	Yes
Lineage to deployment	Yes	Yes	Yes	Yes

Promotion gates: the dataset is promoted like the model

A candidate dataset (a freshly enriched golden_v13, an enriched_retrain_2026_q2 resulting from the Retrain cycle) does not enter production just by being in the bucket. It goes through gates equivalent to the model’s or the prompt’s:

Schema validation — the contract is met. Blocked in CI if not.
Quality validation — a random 5-10 % sample reviewed by a human with a quality score ≥ 4/5. Blocked if the sample fails.
Holdout segregation check — for golden sets and enriched datasets, a hash check against all the other active datasets. Blocked if there is overlap.
Drift check vs previous version — a KS test over the distribution of the input embeddings, or simpler metrics (mean length, segment distribution, the ratio of each label). A warning if the drift is high with no documented cause; a block if it is very high.
Lineage check — the dataset explicitly declares which version it inherits from and what changed. Without that metadata, it does not go in.

Only when all five gates pass is the dataset tagged production-ready and the downstream pipelines that depend on it unblocked (the next Tune, the next product release, the next eval cycle).

The on-premise stack applied

On a generic infrastructure with an RTX 4090 (24 GB VRAM, a development / small-batch profile) and a 4×H100 SXM cluster (80 GB VRAM each, NVLink, training and production inference), data versioning fits without a dedicated GPU for the versioning itself, since versioning lives on CPU + storage, but it does touch the GPU for the drift checks that require embeddings.

Typical topology:

┌────────────────────────────────────────────────────────────┐
│ Object store (MinIO or Ceph) │
│ buckets: /training-sets /corpus-rag │
│ /golden-evals /enriched-retrain │
└────────────────────────┬───────────────────────────────────┘
│
┌─────────────────┼──────────────────┐
│ │ │
┌───▼────┐ ┌────▼────┐ ┌────▼─────┐
│ DVC │ │ lakeFS │ │ MLflow │
│ remote │ │ branches│ │ Tracking │
└───┬────┘ └────┬────┘ └────┬─────┘
│ │ │
└─────────────────┴──────────────────┘
│
┌──────▼──────┐
│ CI/CD gates │
│ (Forgejo / │
│ GitLab) │
└──────┬──────┘
│
┌──────────┴───────────┐
│ │
┌─────▼──────┐ ┌─────▼─────┐
│ RTX 4090 │ │ 4×H100 │
│ (drift │ │ (training │
│ embeds, │ │ + │
│ validates)│ │ serving) │
└────────────┘ └───────────┘

Operational notes:

The object store (MinIO or Ceph) serves both as the DVC remote and as lakeFS storage. One storage plane, two views.
The schema checks and holdout hashing are fast CPU-bound tasks; the CI runner executes them without a GPU.
The embedding drift check needs an encoder; the RTX 4090 handles this without touching the production cluster. A small encoder (BGE-small, E5-small, ~100M parameters) processes 10⁴ examples in a few minutes.
The H100 cluster stays free for training and serving, with no contamination from versioning jobs.

When do you NOT need DVC/lakeFS?

There is an opposite position defended with figures in the continuous fine-tuning post: for small systems with a single team, datasets < 1 GB and a handful of adapters, Postgres + pgvector + an S3 bucket + hashed filenames are enough. The operational complexity of DVC/lakeFS does not pay for itself.

The dividing line is reasonable:

You do not need DVC/lakeFS: a single team, small datasets, few adapters, no multiple products sharing data.
You do need it: multiple teams, datasets > 10 GB, several products sharing a golden eval set, external compliance demanding batch traceability, or an institutionalised quarterly retrain cycle.

Adopting DVC + lakeFS before you need them is overhead. Adopting them six months late is losing six months of lineage irrecoverably.

Seven pitfalls that turn data versioning into theatre

Versioning the data but not the schemas. The content is versioned, the contract changes silently, the system breaks without the versioning capturing it. A Schema Registry is not optional; it is half the problem.
The same S3 path overwritten. “Upload training.jsonl to the bucket” and the next experiment rewrites the file. S3 versioning (if it is enabled) saves your skin, but without an immutable identifier propagated to MLflow you cannot rewind. The correct pattern: training_v3.jsonl or training/2026q2/<sha>.jsonl, never the same name.
A golden eval set without a strict holdout. Without a hash check against training, the model memorises the eval and passes without having learned. It is the LLM equivalent of an exam the teacher announces in advance: everyone passes, nothing has been measured.
Not recording dataset → model lineage. When an incident requires knowing what data a certain model was trained on, the correct answer is a query to MLflow / W&B. If the answer is “let’s ask whoever trained it” (assuming they are still on the team), the lineage does not exist.
DVC added six months late. Adopting versioning in month 1 = a nuisance. Adopting it in month 6 = an irrecoverable loss of six months of datasets that can no longer be reconstructed. The curse of “we’ll add it later”.
lakeFS with branches that never get merged. Parallel branches over the corpus are useful for experimenting; kept indefinitely without merging, operations turn into a graveyard of half-updated branches. Explicit policy: merge or destroy within N weeks.
Schema validation only in production. The contract is validated when the dataset is already in production and the model already trained. By then, the incident has already happened. The validation has to be in CI, before the merge, over the delta the PR introduces.

A dataset’s cycle on one screen

┌─────────────────────────────────────────────────────────────┐
│ Producer (ingest / annotation / retrain loop) │
└────────────────┬────────────────────────────────────────────┘
│
▼ (commit a candidate version)
┌─────────────────────────┐
│ CI gates │
│ - Schema validation │
│ - Quality sampled │
│ - Holdout hash check │ ── fails → PR blocked
│ - Drift vs previous │
│ - Lineage declared │
└────────────┬────────────┘
│ passes
▼
┌─────────────────────────┐
│ DVC tag or lakeFS commit│
│ + MLflow registry │ ← immutable version
│ + Schema Registry │
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ Downstream pipeline │
│ Tune / Eval / Deploy │
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ Production trace │
│ → rewinds to dataset │
└─────────────────────────┘

What we have not covered

At top level, this post leaves out:

Vector store versioning proper: an embedding index is not versioned like a raw dataset because it depends on the embedding model. Changing the embedder rewrites the whole index. It is a different animal and deserves separate treatment (recall, ANN parameters, index branching vs full re-embedding).
Standardised lineage tooling (OpenLineage, Marquez): how to emit and consume lineage events interoperably between systems.
Data quality frameworks (Great Expectations, Soda, Deequ): how to write suites of “expectations” over a dataset and enforce them on every version.
Privacy-preserving versioning: federated learning without centralising the dataset, differential privacy applied to the version that gets distributed.
Contamination between third-party golden sets (HumanEval, MMLU, etc.) and the training datasets of open models: the problem of “the model passes HumanEval because HumanEval is in its pretraining”.

Each one is worth a post of its own when the field justifies it.

References

DVC documentation — versioning workflows, pipelines and remotes.
lakeFS documentation — branching, merging and commits over the bucket.
lakeFS acquires DVC, November 2025 — announcement and combined roadmap.
Confluent Schema Registry and Apicurio — schema contracts for streaming data.
OpenLineage and Marquez — open standard for lineage events.
Great Expectations — data quality expectations in CI.
MLflow Tracking — input datasets as first-class artefacts since MLflow 2.4.
Pachyderm and Quilt — historical alternatives to DVC/lakeFS.
On eval set contamination: “Stop Uploading Test Data in Plain Text” (Magar & Schwartz, 2022) and later work on detecting contamination in pretraining corpora.

Lineage on lo0 — Tech Blog

ISO/IEC 42001: the operations manual for the AI system — how the AIMS fits over the on-premise LLM platform described in this blog

TL;DR

The analogy: the aircraft operations manual

ISO/IEC 42001 in 15 seconds

Distinction from neighbouring frameworks

The seven clauses (Annex SL): the mandatory table of contents

Clause 4 — Context of the organisation

Clause 5 — Leadership

Clause 6 — Planning

Clause 7 — Support

Clause 8 — Operation

Clause 9 — Performance evaluation

Clause 10 — Improvement

The 38 Annex A controls: the AI-specific catalogue

Cross-mapping: 38 controls ↔ blog posts

A.2 — AI policies (2 controls): PARTIAL

A.3 — Internal organisation (3 controls): GAP

A.4 — Resources (6 controls): COVERED

A.5 — Impact assessment (5 controls): PARTIAL

A.6 — AI system lifecycle (4 controls): COVERED

A.7 — Data for AI systems (5 controls): COVERED

A.8 — Information for interested parties (4 controls): COVERED

A.9 — Use of AI systems (3 controls): COVERED

A.10 — Third parties and customer relationships (4 controls): COVERED

The roles defined by the standard

Impact levels and proportionality

The seven minimum AIMS documents

Applied case: the blog’s multi-tenant chatbot → 42001 checklist

Cross-mapping with the EU AI Act, NIS2 and ENS

EU AI Act (Reg. 2024/1689) — seven directly aligned articles

NIS2 (Dir. 2022/2555) — three pillars with overlap

ENS (RD 311/2022)

The five usual certification traps

What we have not covered (upcoming posts)

References

See also

RAG corpus curation: the active librarian who decides what goes in, what goes out and what gets signed

TL;DR

The analogy: the active librarian

The four data artefacts and where the RAG corpus fits

Layer 1 — Schema-validated ingest

Layer 2 — Three-level deduplication

Level A — Exact dedup (SHA-256 hash)

Level B — Near-duplicate (MinHash + LSH)

Level C — Semantic dedup (cosine over embeddings)

Policy on what to do with a duplicate

Layer 3 — PII anonymisation with measured precision/recall

Layer 4 — Anti-contamination against the golden eval set

Layer 5 — End-to-end lineage: from document to trace

The maths that matter

Applied to typical on-premise hardware

The seven traps that kill this stage

What we have not covered (upcoming posts)

See also

References

Data versioning for LLMOps: DVC, lakeFS and the reproducible golden dataset challenge

TL;DR

You are here: Data (with a cross-cutting effect on Tune, Eval and Retrain)

The master analogy: batch traceability in a serious factory

The four artefacts worth versioning (with differentiated demands)

Why Git is not enough

DVC vs lakeFS before the unification

What changed with the November 2025 acquisition

The operational pattern: four-hop lineage

Schema contracts: data versioning without them is an illusion

Golden eval set: the most critical version

Promotion gates: the dataset is promoted like the model

The on-premise stack applied

When do you NOT need DVC/lakeFS?

Seven pitfalls that turn data versioning into theatre

A dataset’s cycle on one screen

What we have not covered

See also

References