Ontologies and knowledge graphs in LLMOps: the Linnaean nomenclature that holds up the six pipeline stages

This post runs through the six stages of the LLMOps pipeline from a cross-cutting perspective: the common nomenclature that makes the stages share a vocabulary. It connects directly to RAG corpus curation (the corpus is curated against an ontology), embeddings (the ontology enriches the embedding with typed metadata), hybrid retrieval (the KG is a fourth channel alongside dense/sparse/multi-vector), evals (golden sets are stratified by ontological class), structured output (JSON Schemas derive from OWL/SHACL), and the three frameworks of ISO 42001, ENS and EU AI Act (each one is a control ontology).

TL;DR

The conversation about ontologies and LLMs has swung between two equally wrong positions over the last three years: either “LLMs already extract knowledge on their own, ontologies belong to the last century”, or “all of RAG has to go and we should build a knowledge graph on top”. The operational reality of mid-2026 is more sober: the ontology is not a replacement for RAG but its common nomenclature, without which the six stages of the LLMOps pipeline work with different vocabularies without knowing it. The corpus is curated without knowing which entity classes exist; the embeddings drill through documents without being enriched with typed metadata; the evals report a global accuracy that hides whole class gaps; the guardrail blocks on word lists instead of formal classification; incident response groups badly because each alert names “the affected asset” its own way; compliance cannot map its controls because ENS, ISO 42001 and the EU AI Act are three ontologies and the system has none. This post takes apart what an ontology is in practical terms (TBox and ABox, RDF and SPARQL, the four OWL 2 profiles: EL for enormous terminologies such as SNOMED, QL for OBDA, RL for rule-based reasoning, DL for the full description logic, plus SHACL for shape validation, SKOS for thesauri, JSON-LD as a viable serialisation), walks the six LLMOps stages showing where the ontology changes operations, reviews the GraphRAG field in 2026 with verifiable data (Microsoft GraphRAG v2 Oct 2025, LightRAG dual-level with incremental updates, HippoRAG 2 with Personalized PageRank, KAG on OpenSPG ontology-grounded), inventories the vertical ontologies actually deployed in production (FIBO, SNOMED CT, schema.org, IEC 81346, GS1, Wikidata, ENS Annex I-II of RD 311/2022, EU AI Act Annex III), settles the viable open source on-prem stack with its licensing caveats (Neo4j Community is GPLv3 with AGPL implications in some features, KuzuDB upstream archived Oct 2025, forks bighorn and ryugraph), describes the five LLM × ontology integration patterns and closes with seven operational traps. The rule of thumb: the knowledge graph is not the answer; shared formalised nomenclature is.

The analogy: Carl Linnaeus, 1735

In 1735 Carl von Linné published the first edition of Systema Naturae. Before Linnaeus, European naturalists had an operational problem: the same species could appear in five treatises under five different Latin names, each a polynomial description of the type “Felis cauda elongata cum maculis nigris in dorso et lateribus”, and two naturalists exchanging letters took months to realise they were arguing about the same animal. Biology was a field of lexicographic noise: impossible to compare observations, impossible to verify replication, impossible to build cumulative theory.

Linnaeus did not discover biology. What he discovered was that the field needed a common nomenclature with three properties:

1. Strict hierarchy. Kingdom → Phylum → Class → Order → Family → Genus → Species. Each level is a class with well-defined subclasses. A property of Felis (a carnivorous diet) is inherited automatically by Felis catus and Felis silvestris without being redeclared.
2. Unambiguous naming. Each species has a single binomial name (Genus + specific epithet) and a single type specimen anchored in a museum. “Felis silvestris” means exactly the same thing in Madrid, Stockholm and Calcutta.
3. Priority rules. If two botanists publish the same genus under different names, the first to register it validly wins. The naming convention is not debated in every paper: there is an explicit meta-level of governance.

After Linnaeus, comparative biology becomes possible. Mendel can talk about Pisum sativum and a Polish botanist knows exactly which plant to grow to replicate him. Darwin can compare Galápagos finches with finches from other islands with no confusion about what “the same kind of bird” means. The change is not one of instrumentation; the microscope had existed since Hooke (1665). The change is one of shared formal vocabulary.

An ontology in computing is exactly this:

When an LLMOps team today says “our corpus is curated, the embeddings are bge-m3 and the evals measure recall@5”, but the question “what proportion of queries about high-category ENS assets are well covered?” has no answer, because no formal class “high-category ENS asset” exists in the system, the problem is pre-Linnaean: the field has not yet given itself the nomenclature that makes each stage comparable.

What an ontology is in operational terms

The word “ontology” carries an unavoidable philosophical family resemblance (Aristotle, Kant’s categories, Quine) that confuses at first. In LLM infrastructure it does not matter: an ontology is a directed graph with types, formally described, over which you can reason, validate and query. What matters are six practical concepts.

TBox and ABox

The distinction used every day. The TBox (from terminology) is the schema: classes, subclass hierarchy, properties, axioms. The ABox (from assertions) is the instances.

# TBox — schema
:Person      rdfs:subClassOf :Agent .
:Employee    rdfs:subClassOf :Person .
:worksFor    rdfs:domain :Employee ; rdfs:range :Organization .

# ABox — instances
:alice       a :Employee .
:alice       :worksFor :acme .

A reasoner checks that the ABox is consistent with the TBox: if you declare :alice :worksFor :acme but :alice is not an :Employee, the reasoner flags the inconsistency. That is the lever: automatic validation of knowledge, which no system based on dense embeddings alone can provide.

RDF and the unit of information

The atomic unit of the Semantic Web is the RDF triple (subject, predicate, object). Every piece of data is expressed as a collection of triples. This gives the paradigm’s most useful operational property: two graphs merge trivially by union. If your system indexes the medical corpus with SNOMED CT and the legal corpus with FIBO, both in RDF, merging them for a query that crosses both domains is literally g1 ∪ g2. In a property graph (Neo4j) this takes more surgery.

The four OWL 2 profiles

People new to the field assume OWL is one thing. It is four profiles with different trade-offs, all W3C Recommendations:

Operational rule: if your team is not going to read a description logics paper every month, do not use OWL 2 DL. Almost all the value is in EL/QL/RL. For large medical terminologies, EL. To reason over existing relational data, QL. For business rules, RL.

SHACL — the validation you actually operate

OWL does reasoning (“given these axioms, what can be deduced?”). SHACL does validation (“given this concrete graph, does it satisfy these shapes?”). In production, SHACL wins because its semantics are closer to the type checking a developer already understands:

:PersonShape a sh:NodeShape ;
    sh:targetClass :Person ;
    sh:property [
        sh:path :nombre ;
        sh:minCount 1 ;
        sh:datatype xsd:string ;
    ] ,
    [
        sh:path :nif ;
        sh:pattern "^[0-9]{8}[A-Z]$" ;
    ] .

Validating an incoming graph against this shape catches :alice :nombre 42 (wrong type), :alice :nif "12345678X9" (wrong format) or :alice a :Person with no name (min count violated). It is JSON Schema for graphs, conceptually. The SHACL 1.2 spec is a W3C draft from 2025; SHACL 1.0 has been in production since 2017.

SKOS — the lightweight thesaurus

Not all knowledge deserves OWL. For controlled vocabularies (thesauri, taxonomies, glossaries) there is SKOS:

:Mamifero a skos:Concept ;
    skos:prefLabel "Mamífero"@es , "Mammal"@en ;
    skos:broader :Animal ;
    skos:narrower :Felino , :Canido .

SKOS does not express formal axioms: skos:broader is not rdfs:subClassOf. It serves to classify content with no pretension of reasoning, which covers 80% of corporate cases. Start with SKOS: most enterprise “ontologies” are in fact thesauri that were promoted to OWL out of fashion and drag unnecessary complexity along.

JSON-LD and SPARQL — the practical surfaces

JSON-LD 1.1 (W3C Rec 2020) is the serialisation that actually gets used in real systems: ordinary JSON with an @context field mapping the keys to IRIs. The schema.org microformat on web pages is JSON-LD. For an LLMOps team, JSON-LD is the natural exchange format with tools and APIs.

SPARQL 1.1 (W3C Rec 2013; 1.2 in draft 2025) is SQL for graphs:

SELECT ?empleado ?empresa WHERE {
    ?empleado a :Employee ;
              :worksFor ?empresa ;
              :pais "España" .
    ?empresa  :sector "fintech" .
}

Every modern triple store speaks it. The federation features let a single query touch several endpoints: SNOMED CT plus your own corporate ontology.

Why it matters for an LLM in production

The romantic promise of 2023-2024 was: “now that we have LLMs we do not need ontologies; the model understands natural language and extracts knowledge”. The operational reality of mid-2026 is more nuanced and rests on four observations anyone with a RAG in production has already made:

1. The LLM has semantic memory but no declared schema. If you ask “which entities of type Person appear in this document?”, it answers something reasonable. If you ask “which people appear and which of them are employees of the customer?”, the answer depends on how the model interprets “employee of the customer” in that context. Without an external schema saying “Employee is a subclass of Person and relates to Organization via worksFor”, coherence between two calls to the same LLM is not guaranteed.
2. Quality varies by domain without the system knowing why. Your RAG has a global accuracy of 78% but fails systematically on queries about derivative financial instruments. Since you have no formal classification of queries by category, the problem is invisible until a customer complains.
3. Compliance demands formal nomenclature. ENS classifies assets along five dimensions (Confidentiality, Integrity, Availability, Authenticity, Traceability) with three levels each. The EU AI Act lists eight high-risk areas in Annex III. Without a formal mapping between your assets and those categories, you cannot audit what you cannot name. The auditor asks “which corpus chunks touch specially protected personal data?” and your system does not have that column.
4. Interoperability between components demands types. Your retrieval returns “relevant chunks”. Your reranker reorders them. Your guardrail filters out the sensitive ones. If each component has its own definition of what a “sensitive chunk” is, the chain breaks at every interface. A shared ontology is the pipeline’s type contract.

The operational consequence: the ontology does not replace RAG. It types it. It makes it auditable, comparable and debuggable. The right question is not “do I need a knowledge graph?” but “at which pipeline stages do I gain if I introduce a shared formal nomenclature?”.

The six LLMOps stages × ontology

Let us walk the six pipeline stages asking what changes in each one when there is an ontology. This is the spine of the post: the lever is not “install Neo4j”, it is introducing types where there was plain text before.

Stage 1 — Data

Corpus curation becomes ontology-driven curation:

• Each chunk is not just “text + embedding”, it also carries chunk:tipoDocumento, chunk:nivelClasificacion, chunk:categoriaENS, chunk:contienePII.
• These types come from an explicit corporate ontology, not from ad-hoc strings written by whichever data engineer was on the morning shift.
• Rule 4 of corpus curation, anti-contamination, benefits: chunks in the golden eval set carry dataset:goldenEval=true declared as a triple; any reindexing that filters on goldenEval=true becomes trivial.
• The PII detector stops being a regular expression and becomes a classifier against the personal data categories thesaurus: identifier, contact, financial, health, biometric. The chunk:pii column is no longer boolean but a list of SKOS categories.

# Ingestion with ontological typing
chunk = {
    "@context": "https://ontology.fibercli.es/v1/context.jsonld",
    "@id": f"chunk:{uuid4()}",
    "@type": "Chunk",
    "tipoDocumento": "ContratoComercial",
    "nivelClasificacion": "ConfidencialMedio",
    "categoriaENS": ["Disponibilidad-M", "Confidencialidad-A"],
    "contienePII": ["IdentificadorFiscal", "Contacto"],
    "embedding": [...],
    "text": "...",
}

Under the JSON-LD context, all those keys resolve to IRIs and are queryable via SPARQL.

Stage 2 — Train / Adapt

Continuous fine-tuning and retrain gain two levers:

• Datasets stratified by class. When production feedback turns into a training dataset, each example comes labelled with the ontological class of the incident that produced it. This lets you sample n examples per class instead of n examples globally, which corrects the model’s coverage gaps.
• Ontology-guided synthetic generation. For classes with few examples in the real corpus, synthetic data is generated against the schema: “generate 50 questions about FIBO:DerivativeInstrument that a trader might ask”. The output goes through structured output validated against the schema’s SHACL shape before entering the dataset.

Stage 3 — Eval

The evals layer changes more than any other. Without an ontology, the eval reports a global accuracy that hides everything:

accuracy = 0.78

With an ontology, it reports a coverage matrix by class:

                        accuracy   n_queries   covered_in_corpus
ContratoComercial       0.82       142         yes
EmpleadoENS-Alto        0.31        18         partial
DerivadoFinanciero      0.74        67         yes
SOAP_3.0_Endpoint       0.05         9         no

The EmpleadoENS-Alto row with accuracy 0.31 exposes a problem that is invisible without stratification. The SOAP_3.0_Endpoint row with accuracy 0.05 and covered_in_corpus=no shows that the class has no corpus at all: before touching the model you have to touch ingestion. A single metric hides; a per-class metric drives action.

This is the rule that LLM-as-judge and evals should always implement wherever an ontology exists: the golden eval set is labelled by class and every metric is reported stratified.

Stage 4 — Deploy

In the LLM inference router the ontology enables:

• Semantic routing by class. Queries that, after a first classification, fall under FIBO:Securities are routed to the adapter fine-tuned on finance; queries under SNOMED:ClinicalFinding go to the medical adapter. Without an ontology, this routing rests on ad-hoc classifiers or fragile lexical heuristics.
• Typed tool calling. The tools the agent can invoke declare their arguments against ontology classes. The cliente_id argument is not a string; it is a :ClienteCorporativo. Before running the tool, the arguments are validated with SHACL. This sharply cuts errors from badly populated arguments.
• Feature flags with a class. The canary becomes “the new model receives 10% of queries in class X” instead of an undifferentiated 10%: it isolates the blast radius.

Stage 5 — Observe

This is where the absence of an ontology hurts fastest in operations. The incident response runbooks require:

• A formal incident taxonomy. IncidenteSeguridad ⊑ Incidente, IncidenteIA ⊑ Incidente, FugaDatos ⊑ IncidenteSeguridad. Without this taxonomy, the five events from last month labelled “model issue”, “data drift”, “pii leak”, “prompt injection” and “hallucination” are neither groupable nor comparable. Keep + Kafka apply deduplication against that taxonomy.
• Typed lineage in the KG. GPU observability plus tracing emits spans with attributes. If those attributes are typed against the ontology (span.input.classification = :ConfidencialMedio), finding every request that touched class ConfidencialAlto in the last hour is a trivial SPARQL query; without an ontology, it is a grep over unstructured logs.

Stage 6 — Govern

Where the ontology becomes unavoidable. Every regulatory framework is an ontology:

• ENS RD 311/2022 Annex I: defines five dimensions (C, I, A, A, T) × three levels (Low, Medium, High). It is an asset classification scheme. Annex II lists 73 control measures with an organisational / operational / protection hierarchy. The ENS technical controls map each control to stack components, and that mapping is a relational ontology.
• ISO 42001 Annex A: lists grouped controls (A.5 policies, A.6 internal organisation, A.7 resources for AI, A.8 assessment, A.9 operation). The AIMS over an on-premise LLM formalises them.
• EU AI Act Annex III: eight high-risk areas. The technical file mappings are a translation from the legal ontology to the system’s technical ontology.

Without an ontology mapping your inventory of assets, datasets, models and endpoints to the classes of those three frameworks, compliance is manual, reactive and breaks with every change to the stack. With the ontology, a model change automatically triggers which controls are affected.

The GraphRAG field in 2026

GraphRAG is the generic name for a family of techniques that build a knowledge graph from a corpus and use it as an additional retrieval layer complementary to the dense / sparse / multi-vector we saw in embeddings. The motivation is that some queries (“what are the dominant themes in this corpus”, “which entities appear connected to customer X in the last six months”) are not answered well by cosine similarity between vectors.

Microsoft GraphRAG

microsoft/graphrag (July 2024, v1.0 Dec 2024, v2.x Oct 2025; any reference to v3 needs checking against GitHub releases before citing). The canonical pipeline:

1. Extraction. An LLM reads the corpus chunk by chunk and extracts entities and relations, so the TBox emerges from the data instead of being declared.
2. Graph construction. The extracted entities are disambiguated, merged and connected through the relations.
3. Community detection with the Leiden algorithm. The graph is partitioned into hierarchical communities.
4. Per-community summaries. For each community, the LLM generates a summary.
5. Local vs global search. Local: neighbourhood traversal for queries about specific entities. Global: map-reduce over community summaries for thematic queries.

The price: building the KG costs on the order of 5-20× more tokens than an embedding pass over the same corpus. For a corpus of 1 million chunks with bge-m3 embeddings (a day of compute on an RTX 4090), a pure GraphRAG typically requires 1-3 weeks of LLM-extractor compute (Qwen2.5-72B or similar). The LazyGraphRAG variant (mid-2025) defers summary generation to query time and cuts the construction cost by an order of magnitude.

LightRAG

HKUDS/LightRAG (HKU, arXiv:2410.05779, October 2024, EMNLP 2025). Practical improvements over canonical GraphRAG:

• Dual-level retrieval. Each query generates both low-level keywords (specific entities) and high-level keywords (themes). The system searches by both and fuses the results. It captures factual and thematic questions in the same pipeline.
• Incremental updates. Insertion of new chunks without rebuilding the whole graph. Canonical GraphRAG requires periodic rebuilds.
• Reported cost: comparatively cheaper than GraphRAG for serving similar queries.

It is the operationally most reasonable GraphRAG when the corpus mutates.

HippoRAG 2

OSU-NLP-Group, arXiv:2502.14802 (Feb 2025; the original HippoRAG at NeurIPS'24). Inspired by the hippocampal indexing model of human memory:

• It builds an open KG and also keeps the original chunks.
• For each query it extracts entities and runs Personalized PageRank over the graph seeded by those entities: the PageRank “marks” the relevant nodes and, transitively, the associated chunks.
• Reported +7% on associative memory tasks over SOTA embedders, with significantly lower indexing cost than GraphRAG, RAPTOR and LightRAG.

It is the most efficient GraphRAG for corpora where “which chunks are relevant to which entities” matters more than “what is the semantic structure of the corpus”.

KAG / OpenSPG

Ant Group + OpenKG, arXiv:2409.13731 (Sep 2024). The key difference from the others: KAG is ontology-grounded. It does not let the LLM invent the TBox; the TBox is declared by the domain (FIBO, SNOMED, corporate ontology) and the LLM only populates the ABox in accordance with that schema. Four pillars:

1. LLM-friendly representation — the schema is exposed in a format the LLM can consume as context.
2. Mutual index between the KG and chunks — each KG node links to the chunks it appears in.
3. Hybrid logical-formal reasoning — it combines the LLM with a declarative rule engine.
4. Semantic alignment — entity disambiguation against the ontological catalogue.

Reported +19.6% F1 on 2WikiMultiHopQA, +33.5% on HotpotQA over the RAG baseline. Deployed in Ant’s e-government and e-health Q&A in production.

KAG is the GraphRAG that works when the domain has a stable ontology (finance, health, government). Canonical GraphRAG wins when the corpus is exploratory and no prior TBox exists.

Others in the landscape

• nano-GraphRAG: a lightweight Python port of GraphRAG; ideal for prototypes.
• Think-on-Graph (ToG) / GraphReader: agents that plan hop traversals over the KG instead of single-shot retrieval. Better at multi-hop QA.
• Neo4j LLM Graph Builder plus LangChain integration: the path of least resistance for companies already running Neo4j.

Vertical ontologies that really get used in production

Three ontologies cover 90% of vertical cases in mid-2026:

FIBO — Financial Industry Business Ontology

EDM Council + OMG, MIT license, OWL DL. The Q1/2026 production release contains 2,446 classes spread across Foundations, Business Entities, Securities, Derivatives, Loans and so on. Used in production for:

• KYC entity resolution: disambiguating legal organisations (fibo-be-le-fbo:FormalBusinessOrganization).
• Classification of financial instruments (fibo-sec-sec-bsk:Basket, fibo-der-drc-cds:CreditDefaultSwap).
• Regulatory reporting: mapping fields against the canonical schema.

For a corporate RAG in finance, FIBO is the type schema any extraction has to satisfy. Without FIBO, two chunks that talk about a “swap” might mean an interest rate swap or a currency one.

SNOMED CT

IHTSDO/SNOMED International. Monthly releases (the May 2026 International Edition published on 15 May). Roughly 350,000+ active concepts in OWL 2 EL. Free licence in member countries (Spain is a member through the CSI / Ministry of Health), commercial outside. In production:

• Assisted clinical coding: the LLM proposes SNOMED codes and the system validates them against the ontology.
• Cross-lingual search in records: Diabetes mellitus type 2 and Diabetes mellitus tipo 2 resolve to the same concept (73211009).
• HIPAA / GDPR health compliance: traceability of what type of clinical data each component handles.

schema.org

CC-BY-SA, ~800 types, native JSON-LD. The ontology of the web. Used in any RAG over public crawls to type Product, Article, Person, Organization from the microformats the corpus already carries embedded.

The others worth keeping on the radar

For a Spanish public sector customer with AI systems, the minimum ontology worth formalising is the union of ENS Annex I + EU AI Act Annex III + ISO 42001 Annex A. That mapping is generated once, kept as a versioned artefact in the AI governance repo and linked from the lineage of every deployed model.

Open source on-prem stack 2026

The implementation landscape splits into RDF triple stores, property graphs and auxiliary tooling.

RDF / SPARQL triple stores

Property graphs (Cypher / Gremlin)

Hybrid vector + graph

• Neo4j 5.x with native HNSW: the vector as a node property, search from inside Cypher. The most integrated option.
• Memgraph + pgvector: two stacks, two operating points.
• Qdrant with a graph payload: not a real graph, but it allows basic k-hop style filters over the payload.

Editors and tooling

• Protégé (Stanford, BSD): the de facto ontology editor. A suite with the HermiT, Pellet and ELK reasoners.
• TopBraid Composer: commercial; useful if it is already in the organisation.
• Atomgraph: LGPL web editor.

Building the KG with an LLM

• GLiNER / GLiREL (Apache 2.0): zero-shot NER and relation extraction. Far cheaper than an LLM-extractor (10-100× fewer tokens).
• REBEL (MIT): joint entity + relation extraction based on BART. SOTA for years, today beaten by LLM-extractors but still reasonable as a baseline.
• LLM-extractor with structured output: vLLM + XGrammar or Outlines enforcing a JSON Schema derived from SHACL. XGrammar has been the default backend in vLLM / SGLang / TensorRT-LLM since March 2026, with <40 µs/token of overhead.

SPARQL clients

rdflib (Python, BSD), the Apache Jena CLI, Comunica (MIT, JS, native SPARQL federation).

Five LLM × ontology integration patterns

Almost everything useful fits into five repeatable patterns.

1. Schema-guided extraction

The LLM emits JSON conforming to a schema derived from the ontology, validated in the decoder with structured output. The output is typed ABox ready to insert as triples:

schema = derive_json_schema_from_shacl("PersonShape.ttl")
# The LLM can only emit tokens that keep the output valid.
extracted = llm.generate(prompt=document, schema=schema)
graph.add_triples(jsonld_to_rdf(extracted))

Cost: practically zero overhead per token with XGrammar; effective elimination of “outputs that do not validate”.

2. Text-to-SPARQL with a semantic firewall

The LLM generates SPARQL; a semantic firewall validates every predicate and class against the TBox before running the query:

sparql_text = llm.generate(prompt=user_query, context=ontology_summary)
query = parse(sparql_text)
for predicate in query.predicates:
    if predicate not in ontology.declared_predicates:
        raise UnknownPredicate(predicate)
result = endpoint.execute(query)

It catches the classic pattern of the LLM inventing a plausible predicate that does not exist in the ontology, before touching the triple store.

3. Hybrid dense + sparse + KG retrieval with RRF

The reranker and hybrid retrieval is extended with a fourth channel: traversal in the KG seeded by the entities extracted from the query. The rankings of the four channels are fused with Reciprocal Rank Fusion:

\text{RRF}(d) = \sum_{c \in \{\text{dense}, \text{sparse}, \text{colbert}, \text{kg}\}} \frac{1}{k + \text{rank}_c(d)}

with k=60 typical. The KG channel covers exactly the queries that break the other three: queries with named entities that dense misreads or that appear rarely in the corpus.

4. Reranking by graph distance

Among the candidates from the first retrieval committee, chunks whose entities lie within k hops in the KG of the query entities are preferred. A practical implementation: add a graph_distance score and fuse it into the reranker:

def graph_distance_score(chunk, query_entities):
    chunk_entities = chunk["entities"]
    distances = [
        shortest_path_length(kg, qe, ce)
        for qe in query_entities for ce in chunk_entities
    ]
    return 1 / (1 + min(distances))

5. Typed tool calling + stratified evals

Tools declare their arguments as ontology classes. Before invocation, the arguments go through SHACL validation. This avoids the classic bug of the agent calling buscar_cliente(cliente_id="the customer who complained yesterday"), a free string where an IRI was expected.

Stratified evals by rdf:type or skos:Concept: each golden set query carries its ontological class as a label, the metrics are reported per class, and global accuracy is complemented by per-class coverage. It is the mechanism evals recommends and the ontology makes operational.

Implications for on-premise inference

The triple store or property graph does not eat GPU: it runs on CPU plus NVMe. What does compete for GPU is the LLM-extractor that builds and maintains the KG.

On the RTX 4090 (24 GB)

A reasonable setup for a PoC and small sites:

GPU 24 GB ┐
          ├─ TEI bge-m3 (dense + sparse + colbert)          │ ~6 GB VRAM
          ├─ vLLM Qwen2.5-7B-Instruct AWQ Q4 (main LLM)      │ ~8 GB VRAM
          └─ Occasional load: vLLM Qwen2.5-7B-Instruct for overnight extraction │ shares VRAM in another window
CPU/RAM  ┐
          ├─ Apache Jena Fuseki (TBox + ABox of the corporate KG) │ ~2 GB RAM per M triples
          ├─ Qdrant (dense + sparse + colbert)                    │ ~3 GB RAM per M chunks
          └─ GLiNER + REBEL for fast batch extraction             │ CPU-only

For corpora of up to a few million chunks, an RTX 4090 does the job by combining GLiNER/REBEL on CPU for bulk extraction (cheap but less accurate) and the LLM on GPU for critical cases.

On the 4×H100 80 GB cluster

H100 #1 (80 GB) ── vLLM Qwen3-72B-Instruct AWQ + Qwen2.5-7B speculative │ main LLM
H100 #2 (80 GB) ── vLLM gte-Qwen2-7B-instruct (embedding 32k ctx)        │ large embedder
H100 #3 (80 GB) ── vLLM Qwen2.5-32B-Instruct (dedicated KG extractor)    │ KG construction + maintenance
H100 #4 (80 GB) ── Hold-out for canary and offline evals                 │ see the canary post

Apache Jena Fuseki cluster (3 CPU nodes + NVMe RAID)
   ├─ Corporate ontology (TBox)
   ├─ ABox (hundreds of millions of triples)
   └─ FIBO / ENS / EU AI Act as named graphs

Qdrant cluster (3 CPU nodes + NVMe)
   ├─ Chunks indexed with triples in the payload
   └─ Lineage towards KG nodes

The H100 dedicated to the KG extractor is the real price of the GraphRAG approach. If the corpus is stable, that H100 can go to offline evals or speculative decoding. If the corpus mutates daily, it is busy keeping the graph up to date online.

The seven operational traps

1. Promoting SKOS to OWL DL out of academic ego. Most “corporate ontologies” are taxonomies that need no description logic reasoning. A SKOS with skos:broader/skos:narrower and skos:prefLabel per language covers 80% of cases. OWL DL only makes sense when there are consistency axioms the reasoner has to verify. Start with SKOS, move up to OWL EL/RL if you need to, avoid OWL DL unless the need is proven.
2. Building a KG over the whole corpus. Canonical GraphRAG applied to 100 million chunks costs as much as training a small model. The correct alternative is HippoRAG 2 / LightRAG / KAG depending on the case, or GraphRAG over the critical subset of the corpus only. The rule: if the construction cost exceeds the annual cost of serving the model, you have picked the wrong tool.
3. A TBox created by an LLM with no governance. Microsoft GraphRAG generates an emergent TBox from the data. For an exploratory corpus that works; for a regulated domain (finance, health, government) the TBox is not discovered, it is declared: FIBO, SNOMED, ENS. KAG is the right choice in those cases.
4. Forgetting KG maintenance when the corpus changes. New chunks introduce new entities. Without an entity reconciliation process (disambiguation, merging), the graph accumulates duplicates of the same entity under different IRIs and quality collapses quietly within six months. LightRAG has primitives for this; canonical GraphRAG requires periodic rebuilds.
5. A JSON Schema out of sync with the SHACL. If the ontology lives in RDF/SHACL and the structured outputs come from a hand-written JSON Schema, they drift apart. The right approach is to generate the JSON Schema from the SHACL with tools such as shacl-to-json-schema and regenerate it in CI every time the shape changes.
6. Neo4j Community licensed badly. GPLv3 means any modification you distribute has to be released under the same licence. If you are going to redistribute a product embedding Neo4j Community, check with legal or use an alternative with a more permissive licence (Memgraph BSL, Apache Jena for RDF, the Kùzu fork bighorn).
7. A compliance ontology not linked to the technical stack. Your ENS / ISO 42001 / EU AI Act mapping lives in a spreadsheet owned by the governance team. Your inventory of models, datasets and endpoints lives in another system. With no formal link between them, no stack change triggers the corresponding compliance review. The mapping goes into the graph, not into the spreadsheet.

Conclusion

An ontology is not an alternative to RAG; it is the nomenclature that makes its pieces comparable. Without it, the corpus is curated with ad-hoc categories, the embeddings drill through documents without being enriched, the evals measure the mean instead of the variance by class, the guardrail blocks on lists instead of types, incident response groups badly because each alert names things its own way, and compliance is a spreadsheet out of sync with the system. The six LLMOps stages are all better when they share a vocabulary, and sharing a vocabulary means formalising a small corporate ontology, aligned with the relevant vertical frameworks (FIBO, SNOMED, schema.org, ENS, EU AI Act), serialised in JSON-LD so the code consumes it without friction, validated with SHACL at every interface and queried with SPARQL when reasoning is needed. GraphRAG in its 2026 variants (Microsoft v2, LightRAG, HippoRAG 2, KAG) is a complementary lever, not the main course: the main course is the shared formal nomenclature. The rest, Neo4j vs Jena, OWL DL vs SKOS, GLiNER vs LLM-extractor, are technical decisions that resolve better once there is clarity about which nomenclature is needed. Linnaeus discovered this in 1735 and biology has not gone back; the LLM field is discovering it in 2026 and will not go back either.

See also

• The six-stage LLMOps pipeline — the master map of the six stages this post cuts across.
• RAG corpus curation: the active librarian — curation becomes ontology-driven curation once a TBox is declared.
• Embeddings in 2026: the three families — embeddings are enriched with typed metadata from the ontology.
• Reranker and hybrid retrieval — the KG is the fourth retrieval channel, fused via RRF alongside dense / sparse / multi-vector.
• Structured output — the JSON Schemas used to build the KG from the LLM derive from SHACL.
• Evals for LLMs — metrics stratified by ontological class are the operational lever the ontology enables.
• LLM tracing with OpenTelemetry GenAI — spans carry attributes typed against the TBox.
• Incident response runbooks — a formal incident taxonomy enables Keep + Kafka deduplication.
• LLM inference router — semantic routing by ontological class.
• Canary, blue-green and shadow — a canary by class reduces the blast radius.
• ENS × ISO 42001 × EU AI Act technical controls — every regulatory framework is an ontology and is mapped as one.
• ISO/IEC 42001: AIMS — Annex A is a control hierarchy that can be formalised as SKOS.
• EU AI Act: the technical file — Annex III is an enumerable classification mappable to the system’s classes.

References

• W3C. RDF 1.1 Concepts and Abstract Syntax. https://www.w3.org/TR/rdf11-concepts/
• W3C. OWL 2 Profiles (EL, QL, RL, DL). https://www.w3.org/TR/owl2-profiles/
• W3C. SHACL — Shapes Constraint Language. https://www.w3.org/TR/shacl/
• W3C. SKOS Reference. https://www.w3.org/TR/skos-reference/
• W3C. JSON-LD 1.1. https://www.w3.org/TR/json-ld11/
• W3C. SPARQL 1.1 Query Language. https://www.w3.org/TR/sparql11-query/
• Edge et al. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. Microsoft Research, 2024. https://arxiv.org/abs/2404.16130
• Microsoft GraphRAG. https://github.com/microsoft/graphrag
• Guo et al. LightRAG: Simple and Fast Retrieval-Augmented Generation. arXiv:2410.05779, 2024. https://arxiv.org/abs/2410.05779
• Gutiérrez et al. HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. NeurIPS 2024. https://arxiv.org/abs/2405.14831
• Gutiérrez et al. From RAG to Memory: Non-Parametric Continual Learning for Large Language Models (HippoRAG 2). arXiv:2502.14802, 2025. https://arxiv.org/abs/2502.14802
• Liang et al. KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation. arXiv:2409.13731, 2024. https://arxiv.org/abs/2409.13731
• OpenSPG / KAG. https://github.com/OpenSPG/openspg
• EDM Council. Financial Industry Business Ontology (FIBO). https://spec.edmcouncil.org/fibo/
• SNOMED International. https://www.snomed.org/
• schema.org. https://schema.org/
• Real Decreto 311/2022, de 3 de mayo, por el que se regula el Esquema Nacional de Seguridad. BOE-A-2022-7191. https://www.boe.es/eli/es/rd/2022/05/03/311
• Reglamento (UE) 2024/1689 (EU AI Act). https://eur-lex.europa.eu/eli/reg/2024/1689
• ISO/IEC 42001:2023 — Artificial Intelligence Management System. https://www.iso.org/standard/81230.html
• Apache Jena. https://jena.apache.org/
• Neo4j Cypher and AI procedures. https://neo4j.com/docs/
• Protégé. https://protege.stanford.edu/
• GLiNER. https://github.com/urchade/GLiNER
• REBEL. https://github.com/Babelscape/rebel