Ia on lo0 — Tech Blog

Real ClickHouse capacity in Langfuse v4: 2,782 bytes per observation, and why almost half of that is not data

Tue, 15 Sep 2026 09:30:00 +0200

Sixth article in the series on operating self-hosted Langfuse v4. The previous five cover what goes into a trace, putting LangGraph in front, two well-known agents instrumented, migrating with no window and the worker queues. This one measures what all of that takes up. Benchmark run on 15 September 2026 against the event schema of version 4.36.0, identical to that of 4.35.0.

TL;DR

Langfuse publishes no capacity figure at all. Not bytes per row, not events per second, not expected size. I searched their code with every reasonable combination and there is nothing. The only quantified thing is the commercial quotas of their managed service, which say nothing about disk.

One observation takes 2,782 bytes of disk. Split across the two event tables: 2,347 in the full table and 435 in the listings one. Measured over two hundred thousand observations shaped the way a real agent produces them.

41.8 % of the full table’s disk is not data, it is indexes. Four full-text indexes add up to 187 MiB against 260 MiB of data. The one covering the input costs 523 bytes per observation, almost three quarters of what the column it indexes costs.

The materialized view trims the text eight times over and saves eighteen per cent of disk. Truncating input and output to two hundred characters leaves the text at 64 MiB of the original 540, and even so the second table adds 18.5 % on top of the first. What survives the trimming is the seventy-eight-column skeleton and a metadata field that is truncated element by element, not as a whole.

In the listings table, metadata costs more than input and output together. 132.6 bytes per observation against 101.8. And its full-text index adds another 111.9, almost as much as the column.

A compression ratio quoted without saying what was compressed is worth nothing. My first pass, with repeated text, gave 38 times. With text of an entropy close to that of natural language it gives 3.68. Same schema, same indexes, same version.

Generations are 11 % of the observations and 65 % of the bytes. Their mean input is 15,655 bytes and the 95th percentile is 55,205, because each one reserialises the full history. It is the quadratic law from the second article in the series, landed on disk.

On deletion, the disk does not go down. A delete stops showing the rows and leaves the space occupied. A second step is needed, applying the deleted mask, and the cron that does it ships disabled, just like the retention one. On top of that the cleaner works table by table, so the second table keeps the rows the first one no longer shows.

And automatic retention is a paid feature when self-hosted. The data-retention entitlement is not in the open edition. Without it, the event tables grow without limit: they carry no expiry clause.

You are here: the sizing you cannot do by reading

The previous five articles describe how version 4 works and how to put it to work. The moment arrives to ask for disk, and there the series runs out of sources.

The official documentation gives a container sizing: two CPUs and eight gibibytes for ClickHouse, in Kubernetes eight of request and sixteen of limit, a hundred gibibytes of disk and three minimum replicas. It is a reasonable starting point and it does not answer the only question that matters, which is how long that disk lasts.

I went looking for it in the code, which is what this series does. I searched for events per second, rows per observation, bytes per row, capacity, benchmark, sizing, across the whole repository and its documentation. There is no figure at all. The closest thing is the ingestion quotas of their managed service, which are commercial API limits, and some working notes on a synthetic load generator that refer to the tables of the previous version and bring no results, only parameters.

So the only honest way out was to measure it. This article is that measurement, with the whole method laid out so that anyone can repeat it and contrast it with their own load.

The analogy: the archive and the index card

A municipal archive keeps case files. Each file has its folder with everything inside, and on top of that the archivist fills in an index card with the header data and the first lines of the subject, so that the file can be searched without pulling out the folder.

Intuition says the cards take up nothing compared with the folders. And that is true if you look only at the paper. It stops being true when the archivist, in order to be able to search by any word, also builds a term file: a box with one card per distinct word appearing in any case file, pointing to where it occurs. That box grows with the vocabulary, not with the number of case files, and ends up taking almost as much room as the case files themselves.

In this setup the folders are the full table, the cards are the listings table, and the term file is the full-text indexes. The article measures the three things separately, because whoever sizes by looking only at the folders falls short by almost half.

The benchmark

I want this to be reproducible, so the method goes in whole.

The schema is the real one. I extracted the ClickHouse migration DDL from the repository, in its single-instance variant. There are not three migrations but six: the three that create the two tables and the materialized view, plus one that adds three ingestion attribution columns, another that adds an n-gram index over the metadata, and another that adds three evaluation columns and two indexes. The result is 78 columns per table, eleven skip indexes in the full table and fourteen in the listings one.

I verified that this DDL is identical between the version I have pinned across the series and the current head of the repository: the diff of the migrations directory is empty.

The engine is ClickHouse 26.9.1. The minimum version 4 demands is 25.12, because of the text indexes the schema needs. I am using a newer version, and that is a difference from an installation that sticks to the minimum.

The data is synthetic and correctly shaped. The benchmark does not invent the rows: the columns and their typical values come from the worker function that builds the record. I write only into the full table, which is what the worker does, and let the materialized view populate the second one. That reproduces the real write amplification.

The topology of each trace reproduces the law we measured in the second article of the series: four fixed observations plus five for each tool call. The number of calls per turn comes from a distribution with a low median and a long tail, capped at fifteen. Generations carry the full message history, which grows with the turn, because that is exactly what the instrumentation does and it is the origin of the quadratic growth in bytes.

Result: 200,012 observations across 22,563 traces, that is 8.86 observations per trace.

And the entropy of the text is calibrated, which is the part that almost ruined the benchmark. My first pass used a repeated paragraph as filler. With that text, the full table compressed 38.1 times and the figures came out beautiful and false. A repeated lorem ipsum is the best possible input for a compressor. I redid the generator with a wide vocabulary sampled from a Zipf distribution and fifteen per cent high-entropy identifiers, calibrated to give the ratio that natural language mixed with identifiers gives.

With that text, the same table compresses 3.68 times. Ten times less. Any ClickHouse capacity figure that does not say what was compressed is noise, and mine would be too without this paragraph.

What one observation takes up

The two tables, after forcing the final merge:

	events_full	events_core
Rows	200,012	200,012
Uncompressed	957.87 MiB	328.73 MiB
Compressed	260.29 MiB	60.34 MiB
Ratio	3.68	5.45
On disk	447.68 MiB	82.92 MiB
Raw per observation	5,022 B	1,723 B
Compressed per observation	1,364.6 B	316.3 B
On disk per observation	2,347 B	435 B

The figure to take away is the last row added up: 2,782 bytes of disk per observation, counting both tables, with one replica and no retention.

And it is worth looking at the distance between the last two rows. The compressed data of the full table is 1,364.6 bytes per observation, but on disk it takes 2,347. The difference, 982 bytes per observation, is the next section.

Where the bytes go

Broken down by column, with what each one costs per observation:

Column	events_full	events_core
`input`	717.5 B	61.5 B
`metadata_values`	389.0 B	132.6 B
`output`	174.7 B	40.3 B
`tool_definitions`	28.0 B	28.1 B
`blob_storage_file_path`	12.6 B	12.5 B
`span_id`	8.1 B	8.0 B
The other 67 columns together	17.3 B	13.3 B

Three things jump out.

The first is that the skeleton is cheap. Sixty-seven columns, most of them empty in a normal observation, cost seventeen bytes between them all. ClickHouse compresses a column of empty strings down to almost nothing. Anyone who feared that the thirteen experiment columns or the telemetry ones would weigh can stop fearing it.

The second is that in the full table the input is half of everything. 717.5 of 1,364.6 compressed bytes. It is the direct consequence of each generation reserialising the history.

The third is the one I was not expecting. In the listings table, metadata costs more than input and output together: 132.6 against 101.8. The reason lies in how the materialized view truncates. It trims input and output to two hundred characters each, but it trims the metadata element by element: an array with forty values can survive with eight thousand characters. The table that exists in order to be light fills up with the one thing its truncation does not control.

The split by observation type

Type	Observations	Mean input	Input p95	Mean output
GENERATION	21,952	15,655 B	55,205 B	1,077 B
AGENT	22,563	2,585 B	6,013 B	1,300 B
TOOL	21,952	319 B	319 B	2,008 B
CHAIN	67,689	413 B	413 B	20 B
SPAN	65,856	272 B	272 B	206 B

Generations are 11 % of the observations and contribute 65 % of the input and output bytes. The 95th percentile of their input is 55 kilobytes, in a benchmark where no conversation goes beyond fifteen tool calls.

This has a sizing consequence worth saying out loud: the size of the database is not determined by the number of requests, it is determined by the depth of the turns. Doubling the users doubles the disk. Doubling the tool calls per turn multiplies it by rather more than two.

The indexes, which are almost half

Here is the finding that changes a sizing exercise the most.

	Data	Indexes	% of disk in indexes
events_full	260.29 MiB	187.31 MiB	41.8 %
events_core	60.34 MiB	22.49 MiB	27.1 %

The marks and the primary key are irrelevant: thirty-two kilobytes and three hundred bytes respectively, with the granule size of 64 mebibytes the full table has declared.

Broken down, and ordered by what they cost:

Index	Table	Bytes per observation
`idx_fts_input_low`	events_full	523.1
`idx_fts_metadata_values`	events_full	319.4
`idx_fts_output_low`	events_full	133.5
`idx_fts_metadata_values`	events_core	111.9
`idx_fts_metadata_names`	both	4.3
`idx_span_id`	both	1.2
The remaining Bloom filters	both	0.1 each

The full-text index over the input costs 523 bytes per observation when the column it indexes costs 717. That is, indexing the input costs almost three quarters of what storing it costs. And the metadata one in the listings table costs 111.9 when its column costs 132.6: eighty-five per cent.

The Bloom filters, on the other hand, are free as far as sizing goes. The four identifier ones and those for model name, experiment and evaluator add up to less than two bytes per observation between them all.

This is not a criticism of the design. Those indexes are what make text search in the interface respond, and without them the experience would be another thing. It is sizing information that is published nowhere: when asking for disk for ClickHouse you have to multiply what the columns say by 1.7.

The materialized view as an amplifier

The second table exists so that listings do not have to touch the large fields. It truncates input and output to two hundred characters and does without the text indexes over them.

The trimming works: the benchmark’s 540.55 MiB of input and output come down to 64.68 MiB, eight and a half times less.

And even so the second table adds 18.5 % of disk on top of the first. For the reasons already seen: the skeleton survives, the metadata truncated element by element survives, and its own text index over that metadata costs another 111.9 bytes.

It is worth keeping in mind because it is counter-intuitive. You would think a table storing two hundred characters of text is an appendix. It is four hundred and thirty-five bytes per observation, almost a fifth of the total, and it is paid always, because the materialized view cannot be switched off without leaving the interface with no listings.

What happens on deletion

The retention cron runs a lightweight delete. I reproduced it: I deleted the observations older than a date, which removed 67,395 rows of the 200,012.

Moment	Visible rows	Rows in the parts	Disk
Before	200,012	200,012	447.68 MiB
After the delete	132,617	200,012	447.68 MiB
After applying the mask	132,617	132,617	299.11 MiB

The delete frees not one byte. The rows stop being visible and the space stays occupied. The disk only goes down at the second step, applying the deleted mask, which frees 33.2 %.

And that second step is run by a different cron that ships disabled, just like the retention one. With the default values, whoever switches retention on in the interface will see the traces disappear from the view and will never see the disk go down, until an ordinary merge drags in the affected parts, which can take however long it takes.

There is a third detail. The cleaner works table by table, with an independent deleter for each one. In my test I deleted in the full table and the listings one kept its 200,012 rows, because the materialized view does not propagate deletes. In a real installation both deleters run, but they are two separate operations that can diverge if one fails.

On top of this you have to add what the fourth article already covered: neither of the two tables carries an expiry clause. There is no automatic expiry in the engine. The whole lifecycle depends on those crons. And per-project retention requires an enterprise edition entitlement, so in the open edition the only retention possible is the one you write yourself.

Projection

With the measured 2,782 bytes per observation, one replica and no retention:

Observations per day	Agent turns per day	Per day	Per month	Per year
100,000	~11,300	0.26 GiB	7.8 GiB	94.6 GiB
1,000,000	~112,800	2.59 GiB	77.7 GiB	945.6 GiB
10,000,000	~1,128,000	25.91 GiB	777.2 GiB	9,456 GiB

The conversion to turns uses the benchmark’s 8.86 observations per trace, which depend on the depth of the agents. With deeper agents, the same turn figure produces more observations.

And there is still a multiplication left. Langfuse recommends three minimum replicas of ClickHouse in Kubernetes. With one million observations a day and three replicas, a year is 2.8 tebibytes of raw disk. The official sizing of a hundred gibibytes per replica covers about thirty-eight days at that rate.

That is the number that is published nowhere and that has to go into the spreadsheet before committing to a service.

What this benchmark does not measure

For the sake of honesty, and because anyone repeating this is going to get different numbers:

The data is synthetic. The shape of the row and the topology of the traces are taken from the code and from the measurements in the previous articles, but the text is generated by a sampler, not by a model. I have calibrated its entropy so that compression resembles that of natural language, and that calibration is the largest source of error in the whole article. A load with a lot of structured JSON will compress better; one with a lot of code or a lot of base64, worse.

It is a single instance, with no replication. In a replicated installation there is a different engine and there are copies.

It is ClickHouse 26.9.1, above the 25.12 minimum. The text index implementations have changed between versions and I have not compared.

It does not measure performance: not ingestion per second, not query latency, not the CPU cost of merges. Disk only. The merge cost of full-text indexes is a topic with a life of its own and is left for the saturation runbook.

And it does not measure object storage, which in version 4 is mandatory and keeps a copy of the original event. That is additional disk this benchmark does not touch.

Checklist

Do not size using the official container sizing: it covers CPU and memory, not disk growth.
Count 2,782 bytes per observation as a starting point, and adjust by measuring your own load.
Multiply by the number of replicas before asking for the volume.
Multiply what the columns suggest by 1.7, because the indexes are 41.8 % of the full table’s disk.
Measure your own installation with the system tables instead of trusting this article: active parts for the total, columns for the breakdown, skip indexes for what is not data.
Watch the metadata, which is the only thing the materialized view’s truncation does not control, because it trims each element and not the whole.
Size by turn depth, not by number of requests.
Switch on both crons, the retention one and the deleted mask one, and check that the disk really goes down after the second.
Reckon on per-project retention requiring a licence, and budget the work of writing it if you do not have one.
Remember that the tables have no expiry in the engine: with no cron, they grow for ever.

Traps and things that are not what they look like

Langfuse publishes no capacity figure at all, neither in its code nor in its documentation.
A compression ratio without saying what was compressed means nothing: the same schema gives 38 times with repeated text and 3.68 with realistic text.
41.8 % of the full table’s disk is indexes, not data.
Indexing the input costs almost as much as storing it: 523 bytes per observation against 717.
The 78-column skeleton is irrelevant: seventeen bytes per observation across the sixty-seven small columns.
In the listings table metadata costs more than input and output together.
The materialized view’s truncation trims each element of the metadata, not the whole array.
The listings table is not an appendix: it adds 18.5 % of the first table’s disk despite storing two hundred characters.
A delete frees no disk. The mask has to be applied, and that cron ships disabled.
The retention one also ships disabled out of the box.
The cleaner works table by table; the materialized view does not propagate deletes.
The event tables carry no expiry clause.
Per-project retention is a paid feature when self-hosted.
Generations are 11 % of the observations and 65 % of the bytes.
Bloom filters are free as far as disk goes; the full-text ones are not.

Closing

What I take from this benchmark is not the figure, although the figure was needed. It is that the split is not where you look for it.

When somebody sizes a trace database, they look at the size of the prompts. It is the natural thing to do: they are the large fields and they are the ones you see in the interface. And it turns out that in this architecture the prompts are half the compressed data, and the compressed data is a little over half the disk. The rest is search structures nobody counts because they do not show up when you look at a row.

There is a pattern there that goes beyond this tool. An observability database is, by definition, a database queried in unforeseen ways: by free text, by user, by session, by whatever is needed when something goes wrong at three in the morning. That ability to search by anything is paid for in disk, always, and it is paid at write time and not at search time. Langfuse’s design pays it fairly explicitly, with four full-text indexes declared in the migration, in plain sight for anyone who wants to read them.

What was missing is for somebody to put the number alongside. Now it is there, with its margin of error and its method on the table, so that the next person to measure it can tell me where I got it wrong.

The series: the ten articles

What goes into a trace: version 4’s data model, limits, precedences, scores, masking and indexes.
Putting LangGraph in front: instrumenting an agentic platform and the measured cost of one turn.
Two well-known agents instrumented: Open Deep Research and GPT Researcher, with the measured figures of a real request.
Migrating from version 3 to 4 with no window: the three write-mode steps, the resumable background migrations and where the rollback point of no return sits.
The worker queues: the map of all thirty-nine, which pool to dedicate to each group, the per-queue switches, sharding and concurrency.
Real ClickHouse capacity and cost (this article): how to measure bytes per observation with the system tables, the difference between the full table and the listings table, and the merge cost of full-text indexes.
Retention, deletion and data protection: why a deletion does not free disk, the mask cleaner that ships disabled, the pending deletion queue and the S3 lifecycle that has to be implemented by hand.
Backup and cross recovery: restore ordering across Postgres, ClickHouse and object storage, what each mismatch breaks, and how far event replay goes.
Saturation runbook: what to alert on from the queue metrics, the stuck probes, draining through the readiness endpoint and the dead letter queue.
Getting the data out: the blob storage integration to Parquet, batch exports and the metrics API, to build the data lake.

Sources

Langfuse, migraciones de ClickHouse (packages/shared/clickhouse/migrations/canonical/, ficheros 0039 a 0047): https://github.com/langfuse/langfuse.
Langfuse, escritura del worker en ClickHouse (worker/src/services/IngestionService/, worker/src/services/ClickhouseWriter/): https://github.com/langfuse/langfuse.
Langfuse, limpiador de retención por lotes y limpiador de máscaras de borrado (worker/src/features/batch-data-retention-cleaner/, worker/src/features/deleted-mask-cleaner/): https://github.com/langfuse/langfuse.
Langfuse, guía de autoalojamiento y requisitos de infraestructura: https://langfuse.com/self-hosting.
ClickHouse, tablas del sistema parts, columns y data_skipping_indices: https://clickhouse.com/docs/en/operations/system-tables.
ClickHouse, índices de salto y de texto completo: https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/mergetree.
ClickHouse, borrados ligeros y máscara de borrado: https://clickhouse.com/docs/en/guides/developer/lightweight-delete.

deepagents on a cluster of your own: the SDK is free, the server is not, and what you have to build in between

Tue, 15 Sep 2026 07:00:00 +0200

Opening of the agentic platform vertical. The previous articles dealt with the pieces separately: the inference gateway, its second door, what a fleet of agents costs and isolation. This one deals with what goes on top. Verified against deepagents 0.7.14 and the LangGraph repository, commits of 14 September 2026.

TL;DR

deepagents is not a runtime. It is a factory that assembles middleware and delegates to LangChain’s agent factory. It returns a compiled LangGraph graph and nothing else. The repository’s own documentation says so without decoration: it introduces no new runtime. That is good news for whoever deploys it, because it runs wherever Python runs.

The planning tool no longer exists. It disappeared in 0.7.0 as a breaking change, along with the state channel that backed it. Almost everything written about this harness still describes it as one of its three legs. Today you have to ask for it explicitly, and you have to ask for it in each subagent too.

With the default configuration, the agent’s files live inside the checkpoint. The state backend stores the full content, binaries in base64, in the same place the conversation is persisted. Changing that is one line and it is the first architecture decision of the deployment.

The database grows per loop step, not per turn. Four tables, one row per superstep, one per channel and version, another per task write. And of the four cleanup operations the base class declares, the Postgres implementation has only one: delete an entire thread. The other three raise not-implemented.

Pruning by hand can empty the history silently. The code itself warns that a partial delete breaks the delta chain and leaves the channels rebuilding empty without raising any error. The only safe pruning is by whole thread.

Tenant isolation is the thread identifier, and there is no owner check. Whoever can send a thread identifier reads that thread. The on-disk filesystem backend fixes its root when the agent is built, so it separates nobody. And the documentation’s example for computing the per-user namespace does not work outside the managed platform.

There is a configuration trap that leaves a short-window model uncompacted. If the model does not expose its window, the compaction threshold falls back to 170,000 tokens. Served on a vLLM with a 32,768 window, the agent blows up on context length long before it compacts. The failure is silent and is fixed by passing the profile by hand.

Observability without the platform does exist, and it goes through a package that is not obvious. LangChain’s core does not emit OpenTelemetry: zero references. The exporter lives inside the LangSmith SDK, and there is a mode that emits over OTLP only, without talking to the service. It is the road to a Langfuse of your own, and it is worth knowing where it passes.

And here is where the free part ends: the server. The SDK, the graph engine and the checkpointers are MIT. The server that publishes them over HTTP carries an Elastic 2.0 licence, asks for a commercial licence key for production, adds Redis to the stack and reports metadata to an external endpoint unless there is an isolation agreement. The development tool is explicitly for development.

What is left is a list of things you have to write. Per-thread authorisation, retention, a run queue, cancellation, clean draining and migrations. None of them is hard. All of them are work, and none of them appears in the twenty-line front-page example.

You are here: the harness arrives after the model

The sequence repeats on every project. First you set up inference, which is the visible part. Then the gateway, because you need keys and budgets. Then observability, because nobody knows what is going on. And when all of that works, somebody asks why the assistant cannot do multi-step tasks, and the word agent enters the scene.

At that point the temptation is to write the loop by hand, and for two weeks it looks like a good idea. The loop is easy. What is not easy is everything you discover afterwards: what happens when the context fills up, where the intermediate files are stored, how you resume a task that was left half done, how you ask a human for confirmation without losing state.

An agent harness is exactly that: the packaged answer to those questions. And choosing one looks like a library decision, about the size of choosing an HTTP client. It is not. It decides where your users’ state lives, which database grows, what can be isolated and what cannot, and in the case at hand, whether there is a commercial licence waiting at the end of the road.

This article looks at deepagents with those eyes. It is not a usage guide, and I do not assess whether its agents are any good. I look at what it forces you to build.

The analogy: the scaffolding and the crane

A building site needs scaffolding and it needs a crane, and they are not the same thing even though both are metal structures surrounding the building.

You put up the scaffolding yourself with parts you bought. It adapts to the façade, it can be extended, and the day the work finishes you take it down and put it away. Nobody charges you for using it once it is bought.

The crane is another matter. You hire it, it comes with an operator, it has a serial number and you have to notify somebody when it goes up. Without a crane the work still moves forward, more slowly and with more people, but it moves forward.

Here the scaffolding is the SDK and the graph engine, and it belongs to whoever buys it. The crane is the server that exposes the agent over HTTP, and that one comes with a contract. This article is about where exactly the line between the two sits, because you cannot see it in the brochure, and about how much work can be done without a crane.

What deepagents actually is

The first thing to get out of your head is that it is a system. It is a function.

The main factory assembles a list of middleware and calls LangChain’s agent factory, which is what builds the graph. What it returns is a compiled graph, with an added configuration on top that raises the recursion limit to a very high number and adds an integration tag in the metadata. The repository’s own architecture document states it: no new runtime is introduced.

Compiled with the default options, the graph has the nodes you would expect from a reasoning-and-action loop: start, a prior node that repairs incomplete tool calls, the model node, the tools node and the end. The conditional edges run from the model to tools, to the model itself or to the end. There is nothing more. Everything that looks like a system is middleware that adds nodes only when it declares hooks of its own.

The tools that ship by default are the filesystem ones, plus the subagent delegation one: list, read, write, edit, delete, search by pattern, search by content, execute, and the task tool.

The good consequence of this is portability. If something is a LangGraph graph and nothing else, it runs in any Python container, with the checkpointer you choose, against the model you choose. There is no hidden service behind it.

What is no longer there

It deserves its own section because it contradicts almost everything published. The planning tool, the one that wrote a task list and kept it in a state channel, was removed in 0.7.0 as a breaking change declared in the changelog. The tool is gone, the channel is gone, and the prompt fragment that went with it is gone.

It survives as LangChain middleware, and you have to ask for it by hand. With one detail that bites: you have to ask for it in each subagent too, because each subagent assembles its own stack. All that remains from the previous stage are archaeological leftovers, such as the state key that is still on the list of keys excluded when passing context to a subagent.

The plan, today, rests on files. Which is consistent with the rest of the design, and is another reason to look closely at where those files live.

Where state lives

This is the first architecture decision and the one that is most expensive to get wrong.

The agent’s state has two large channels. Messages, with a reducer of its own that deduplicates by identifier and treats deletions as tombstones. And files, with its own reducer where a null value means deleted. Both use a channel type that writes deltas and stores a full snapshot every fifty updates, and the code comment explains why: to take checkpoint growth from quadratic to linear with respect to the number of messages.

That channel type is marked beta in its own code, with a warning that the on-disk representation may change. Worth knowing before you rest a platform on top of it.

The backends

There is a common interface and several implementations. The ones that matter for deciding:

Backend	Where the files live	Persistence
State	Inside the graph state, that is, inside the checkpoint	Per thread, in the checkpointer’s database
Filesystem	In a real directory anchored to a root	Wherever you mount the volume
Store	In LangGraph’s store, with a namespace	Crosses threads, with a time to live
Composite	Routed by path prefix among the above	Mixed

The default backend is the state one. That is: by default, everything the agent writes ends up inside the checkpoint, and binaries go base64-encoded inside the same JSON. For a demo it is convenient, because you have nothing to set up. For a platform with real users it is a sizing decision taken by omission.

There is a detail that rounds the matter off. The filesystem middleware automatically evicts large tool results to a file when they exceed a token threshold, so that they do not occupy the context window. It is a good mechanism. But with the state backend, evicting to a file means moving the content from one place in the checkpoint to another place in the same checkpoint. It leaves the model’s window and it does not leave the database.

What grows in the database

The Postgres checkpointer creates four tables: one for migrations, one for checkpoints, one for blobs by channel and version, and one for writes by task. The primary keys all start with the thread identifier, and there is an index per table on that column.

What you have to internalise is the rhythm. One checkpoint row is written per engine superstep, not per conversation turn. A turn with four tool calls is several supersteps. None of that is recycled on its own.

And here comes the uncomfortable part. The checkpointer’s base class declares four maintenance operations: delete a thread, delete by run, copy a thread and prune with a strategy. In the Postgres implementation only the first is implemented. The other three inherit the not-implemented exception from the base class. Deleting a thread is three delete statements by identifier, and that is it.

In other words: retention is yours to build. And with a care that the code itself documents and that is worth quoting in full, because it is the kind of warning you read after the incident. A naive pruning that removes intermediate checkpoints and their writes can cut the delta chain; the surviving checkpoint is rarely a snapshot point, so its channels would rebuild empty without any error being raised.

Translated into operations: the periodic sweep deletes whole threads and only whole threads. Never loose rows. And since the checkpoints table has no date column, to know which thread is old you have to keep count outside or rely on the checkpoint identifier being a time-sortable UUID.

There is a variant of the checkpointer that stores only the last state and retains no history, which caps growth at the root. With one important reservation to verify before using it: it is not clear that it is compatible with the delta channels this harness uses by default for messages and files. Combining the two without checking is asking for a silent problem.

The store, which does have cleanup

Curiously, the sibling piece is better resolved. The store has a configurable time to live, with refresh on read, skipping of expired entries and a sweep interval, and the sweep is a delete by expiry date.

Two warnings. The sweeper is a thread inside the process, so with several replicas there are several sweepers competing for the same delete; it is idempotent, but it is free contention and you avoid it by switching it off and putting the delete in a scheduled cluster job. And the store’s semantic search requires the vector extension in the database, which constrains the managed Postgres image, which normally does not carry it.

The isolation that does not exist

If the platform is going to have more than one tenant, this is the section that decides the deployment topology.

The only thing separating one thread from another is the thread identifier. It is the first column of the primary key in all three tables. There is no owner check anywhere in the checkpointer. Whoever manages to send a thread identifier in the call configuration reads that thread. Authorisation is the responsibility of the layer you write on top, and since that layer also has to be written, it is worth noting down now.

By backend, the situation differs:

The state one inherits thread isolation, which is enough if the HTTP layer validates that the user owns the thread.
The store one computes the namespace per call, through a function that receives the runtime context. It is the correct option for multi-tenant.
The filesystem one fixes its root in the constructor, when the agent is built. There is no path in the code that recomputes it per request, per thread or per tenant. Sharing one deployment between tenants with this backend means everybody sees the same directory.

On that last point, do not confuse it with the protection that does exist. Virtual mode, which now comes enabled by default, blocks paths that escape the root. That protects against directory traversal. It does not protect against the neighbour. The project’s own threat document admits it: that mode exists to support the composite backend’s prefix routing, not as a security boundary.

There is also a documentation trap worth flagging because it costs an afternoon. The example the documentation proposes for computing the per-user namespace reads the identity from a server-information field of the runtime context. That field is annotated in the LangGraph code as metadata injected by the managed server, and null when running open-source LangGraph without managed deployments. On a cluster of your own with an HTTP layer of your own, that example fails. What you have to use is the context you fill in yourself, with the identifier that comes validated from the identity provider.

And one more boundary that gets crossed without warning: the memory and skills middleware interpolate file content into the system prompt as is, without sanitising. If two tenants share a directory that serves as the source for that, the first writes instructions that the second’s agent executes. It is prompt injection through shared storage, and it is avoided with the same measure as everything above: do not share the filesystem backend between tenants.

The topology conclusion is short. Real multi-tenant with files means one deployment per tenant, with its namespace, its volume and its quota, or else the store backend with a namespace computed per request. There is no third way.

The model, and the trap you need to know about

The default model is a specific commercial provider, and it is marked deprecated: passing an empty model warns and will stop working. You can inject any LangChain chat model, which is what anyone serving their own weights will do.

With two integration details that are not obvious.

The first. To point at an OpenAI-compatible endpoint, which is how a vLLM behind a gateway looks, you have to pass the already-built instance, not the string with the provider prefix. With the string, a provider profile is applied that forces the use of the responses API, which a vLLM normally does not implement. Passing the built object applies no profile and the agent stays clean.

The second is the trap, and in my view it is the most likely configuration failure in the whole deployment.

The compaction middleware computes its thresholds from the model profile. If the model declares its input window, it compacts on reaching 85 % of the window and keeps 10 %. If it does not declare it, it falls back to a fixed value: 170,000 tokens.

The chain that leads to not declaring it is the usual one in a deployment of your own. The OpenAI client resolves the profile by looking the model name up in a static table of the provider’s models. A model served as qwen3-30b-a3b is not in that table, so the profile comes out empty, and empty becomes null. The method that assigns it swallows any exception, so there is no warning.

Result: a vLLM with a 32,768-token window will not compact until 170,000. The server will return a context-length error long before that, and it will do so intermittently, only when conversations get long. There is a safety net that catches the error and trims, but that is reactive recovery after a failed call, not planning.

The fix is one line, because the profile is a public field and only fills itself in when it is empty: you pass it when building the model, with a value below the real window to leave room for generation. And it deserves an assert at start-up, because it is the kind of thing nobody looks at until it fails.

A related sizing note. With the default thresholds and a 32,768 window, compaction would trigger at 27,852 tokens, while eviction of large tool results is fixed at 20,000. That is, a single tool result can occupy almost the whole budget before anything evicts it. With 128,000 windows the defaults are proportionate; below 64,000 you have to lower those two thresholds by hand.

The tools, and the MCP that is not where it seems

This harness has its filesystem tools and its delegation tool, and the rest are passed as an ordinary list of LangChain tools.

The fact that matters to anyone with an MCP gateway in place: the SDK does not integrate MCP. There is no reference to the protocol in the package, nor any dependency on adapters. The integration lives in the command-line tool that accompanies the project, which does depend on the MCP adapters for LangChain, resolves configurations in the style of the desktop clients, and turns each remote tool into a LangChain tool that then goes through the ordinary list.

So the path exists and is proven, but you have to walk it yourself: load the gateway’s tools with the adapters and pass them in the list. Which, incidentally, fits well with what we already know about the catalogue: the credential-based filtering the gateway does is applied in the listing, so each agent receives the subset that belongs to it without the harness having to know anything.

The subagents

They are declared in three ways and invoked with a task tool that takes a description and a type. What you need to know to design with them:

The subagent receives the parent’s state except the messages and a few private keys, and it starts with a single human message which is the task description. Files are inherited, and it shares the same backend object, so parent and child work on the same logical filesystem. The result comes back as a tool message with the structured response or the child’s last text.

Depth is effectively one level. Verified by compiling the graph: the stack assembled for a declarative subagent does not include the subagent middleware, so the child does not have the task tool and cannot delegate in turn.

And there is no concurrency limit of its own. The parallelism is that of the engine’s parallel tool calls, and the task tool’s prompt explicitly invites launching them in parallel. Anyone with a per-tenant GPU quota will want to put the brake where it belongs, because it is not here.

Code execution

The sandbox protocol is well defined and the packaged implementations are nearly all paid services: four commercial providers, plus an embedded JavaScript interpreter and a sandbox from the platform vendor itself.

That leaves a local shell backend that is worth looking at closely before considering it. It runs the command with the system shell, and its own docstring lists what it does not do: no isolation, no process separation, no resource limits, and it lists production and multi-tenant environments as inappropriate use cases. It also has an option to inherit the process environment which, enabled, hands the model every variable, including the gateway credentials and the database connection string. It comes disabled, and it is an easy foot to shoot.

The reasonable way out on a cluster of your own is to implement the protocol yourself, and it is smaller than it looks. The sandbox base class requires four things: an identifier, execute, upload files and download files. Everything else, list, read, write, edit, delete and search, is built by the base class on top of execute. Looking at one of the commercial implementations, it is on the order of a hundred and fifty lines.

One image requirement that constrains the Dockerfile and is worth knowing beforehand: the base class helpers inject small encoded Python scripts to resolve searches and checks, so the sandbox image needs Python and a POSIX shell or half the calls fail.

On how to implement it, the shape that fits a Kubernetes of your own is an ephemeral container per session with a bounded time to live, resource limits, no service account token mounted and a network policy that only allows egress towards the inference gateway. Execute is resolved against the cluster’s exec API, and upload and download through the same channel. The alternative with less code is an isolated runtime class for the agent pod, one pod per tenant, and the shell backend inside; you lose multi-tenancy in a single pod and you have nothing to write.

Observability, and the package it goes through

Here there is a fact that surprises and that is worth being clear about before designing the trace pipeline.

LangChain’s core does not emit OpenTelemetry. Zero references in the whole package. Its native tracer talks to the vendor’s observability service and that is that.

The OTLP exporter does exist, but it lives inside that service’s SDK. And there is the good news: that SDK has a tracing mode that emits only over OTLP, without talking to the service, reading the destination and headers from OpenTelemetry’s standard environment variables. The transport is HTTP with protobuf, which is exactly what a self-hosted Langfuse accepts on its ingestion route.

That is: to take the agent’s traces to your own observability you have to install the vendor’s package and ask it not to talk to the vendor. It works, it is supported and it is documented, but it is a dependency you have to declare in the supply chain analysis, not a configuration detail. And if the OpenTelemetry packages are missing, it emits a warning and silently stops tracing, so start-up should check for it.

There is the alternative of the callback handler that Langfuse itself publishes, with fewer pieces. The practical difference is correlation: over the OTLP route the agent’s spans share a trace identifier with the gateway’s and the inference engine’s, and you see a whole request end to end. Over the callback route, you do not. For anyone who already has a collector deployed, the OTLP route is the one that pays.

The server: here is where the free part ends

Everything above is an architecture decision. This is a licence decision, and it is the one that gives the article its title.

The harness SDK is MIT. The graph engine is MIT. The memory, SQLite and Postgres checkpointers are MIT. All of that runs anywhere, with no key, no registration and without calling anybody.

The server that exposes a graph over HTTP is another matter. The package that implements it carries an Elastic 2.0 licence, whose relevant clauses forbid offering the software to third parties as a managed service and forbid modifying or circumventing the licence key functionality. The command-line tool’s own command prints the conditions: for local development it asks for a service API key, and for production use it asks for a licence key in an environment variable.

There is also usage reporting. The server code contains a beacon endpoint to which it sends metadata, with a header carrying the licence key. The self-hosting documentation confirms it from the other side, listing among the requirements network egress to that domain for licence verification and usage reporting if not running in isolated mode. Isolated mode is a contractual variant, not a box you tick.

And it drags infrastructure along: besides Postgres, it asks for Redis.

The development tool, which is the one that appears in every tutorial, describes itself as development mode with hot reload and an in-memory server. It is not a production server and it does not claim to be.

None of this is a reproach. It is a legitimate and fairly common business model: permissive core, commercial control plane. What is not legitimate is finding out late. Anyone building a platform with a sovereignty requirement, with no dependencies on external services and with the argument that the whole stack is open, has to know that the piece that turns the graph into a service does not meet that requirement.

What you have to build

The way out is to write the HTTP layer yourself, and the good news is that it is boring. A service that compiles the graph once with the checkpointer pointing at the database, and exposes an endpoint that validates the identity provider’s token, derives the thread identifier and the tenant context, and invokes the graph. Event streaming if needed.

What you additionally have to write, and which the commercial server gave you ready-made:

Per-thread authorisation. The engine does not have it. It is the most important and the easiest to forget, because nothing fails if it is missing.
Retention and deletion. A scheduled job that deletes whole threads, never loose rows, for the reasons in the state section.
Migrations. Schema setup has to be called explicitly and needs data-definition permissions, so it goes in a separate job with a different role from the runtime one. And since it creates indexes concurrently, it cannot go through a connection pooler in transaction mode; point it at the direct write service.
Health probes and draining. The engine has a cooperative draining mechanism, but wiring it to the container’s termination signal is your own work.
Run queue and cancellation, if you need long background runs. This is where the absence is most felt, and the reasonable answer is a queue on the same database rather than another system.
Durability decided deliberately. There are three modes: persist before the next step, persist in parallel, or persist only on exit. The last one is no good on Kubernetes: if the pod dies, the whole turn is lost.

On that last point it is worth being explicit, because it affects tool design. The engine stores each task’s writes as soon as it finishes, and on resume it reapplies those that were already there and does not repeat them. A tool that finished is not run again. A tool that was in flight when the pod died is, because its write never landed. The semantics are at-least-once, so every tool with an external effect has to be idempotent or carry an idempotency key. There is no exactly-once.

And the same applies to interrupts for human confirmation: on resume, the node is re-executed in full, so any side effect that sat before the interrupt inside the same node is repeated. The rule is that the interrupt goes first.

How it fits with activity logging

Two things about this setup touch compliance head on, and both come out of earlier sections.

The first is that the checkpoint is, de facto, a repository of personal data. It contains the whole conversation and, with the default backend, the files the agent has written. If the platform serves identifiable people, that database needs encryption, retention of its own and separate access control, exactly like the gateway’s spend table. The difference is that here there is no redaction switch that helps: the state is the state.

The second is traceability. An agent that delegates to subagents and executes tools produces a chain of actions that you have to be able to reconstruct, and the only place where that chain is complete is the trace. Traces going out over OTLP to your own observability stops being a technical preference and becomes the logging mechanism. It is worth designing it that way from the start, and not adding it afterwards.

As always in this series, the annex’s specific codes have to be checked against the current text before taking them into a compliance document; the approach is developed in the technical controls article.

Checklist

Decide the filesystem backend before the first demo, because the default one puts the content in the database.
Pass the model profile by hand with the real window, and check it at start-up.
Lower the eviction thresholds if the model’s window is below 64,000 tokens.
Build the model as an instance, never as a string with a provider prefix, if there is an OpenAI-compatible endpoint behind it.
Write per-thread authorisation before opening the service to more than one user.
One deployment per tenant if you use the filesystem backend; a namespace computed per request if you use the store one.
Compute that namespace from your own context, not from the server-information field, which is null outside the managed platform.
A scheduled retention job that deletes whole threads and only whole threads.
Schema migrations in a separate job, with a role of their own and against the direct write service.
Synchronous durability, and tools with external effects made idempotent.
Traces over OTLP with the mode that does not talk to the vendor’s service, and a start-up check that the packages are there.
Put a concurrency limit on subagents if there is a GPU quota involved.
If you need code execution, implement the sandbox protocol against an ephemeral container; the local shell backend is advised against by its own documentation.
Note in the supply chain analysis the hard dependencies on commercial providers that the package drags in even when unused.

Traps and things that are not what they look like

The planning tool was removed in 0.7.0. You have to add it by hand, and in each subagent too.
The default backend stores files inside the checkpoint, binaries included in base64.
Evicting a large result to a file does not get it out of the database with that backend, only out of the context window.
One checkpoint is written per superstep, not per turn.
Of the four declared cleanup operations, only delete-a-thread exists in Postgres.
A partial pruning can empty the history without raising any error.
The delta channel is in beta and its on-disk representation may change.
There is no owner check in the checkpointer. The thread identifier is not a credential.
The filesystem backend’s root is fixed when the agent is built and it separates no tenants.
The documentation’s example for the per-user namespace does not work outside the managed platform.
The filesystem middleware and the subagent middleware cannot be removed.
If the model does not declare its window, compaction does not trigger until 170,000 tokens, silently.
Passing the model as a string with a provider prefix forces an API that a vLLM does not implement.
The SDK does not integrate MCP; the integration lives in the command-line tool.
Subagents do not delegate: depth is one level.
LangChain’s core does not emit OpenTelemetry; the exporter lives in the vendor’s service SDK.
The HTTP server carries an Elastic 2.0 licence, asks for a licence key for production and reports usage unless there is an isolation agreement.
The development tool is for development, with an in-memory server.
Resumption is at-least-once for in-flight tools.

Closing

The question this article started with was whether deepagents fits into a platform of your own. The answer is that it does, and that it fits better than I expected, precisely because it is less than it appears. A middleware stack on top of a graph is easy to host, easy to understand and easy to replace. What does not fit is the piece next to it.

That asymmetry is the pattern I have been seeing all year across the ecosystem, and it deserves naming. The core is published under a permissive licence because the core is where you compete for adoption. The control plane is published under a restrictive licence because the control plane is where you charge. It happened with the gateways, it is happening with observability, and it happens here. Anyone building a sovereign platform cannot assess a whole project by the licence of its main repository: you have to look package by package, and look at which of them is the one you are going to need the day the thing has users.

The good part is that the bill, in this case, is paid in work and not in money. What you have to build to do without the commercial server is an HTTP service with no mystery to it, a retention job, a migration job and an authorisation layer. None of that is research. All of it is a week of an engineer who knows what they are doing, and in exchange you are left with a system that deploys the same way in a school as in a classified datacenter, with no internet egress and with nobody counting runs on the other side.

What I do recommend is making that list before the first demo and not after. Because the front-page example works in twenty lines, and the twenty lines have the default backend, the model without a profile and no authorisation at all.

Sources

langchain-ai/deepagents, código y documento de arquitectura: https://github.com/langchain-ai/deepagents.
langchain-ai/deepagents, registro de cambios de la rama del SDK (eliminación de la herramienta de planificación en la 0.7.0): https://github.com/langchain-ai/deepagents/blob/main/libs/deepagents/CHANGELOG.md.
langchain-ai/langgraph, motor, canales y checkpointers: https://github.com/langchain-ai/langgraph.
langgraph-checkpoint-postgres, esquema de tablas y operaciones de mantenimiento: https://github.com/langchain-ai/langgraph/tree/main/libs/checkpoint-postgres.
langgraph-checkpoint, clase base del checkpointer y aviso sobre poda con canales de deltas: https://github.com/langchain-ai/langgraph/tree/main/libs/checkpoint.
LangChain, despliegue de servidor autónomo (requisitos de licencia, Postgres, Redis y salida de red): https://docs.langchain.com/langsmith/deploy-standalone-server.
LangChain, trazado con OpenTelemetry: https://docs.langchain.com/langsmith/trace-with-opentelemetry.
Elastic License 2.0: https://www.elastic.co/licensing/elastic-license.
Langfuse, integración nativa OpenTelemetry (ruta de ingesta y autenticación): https://langfuse.com/integrations/native/opentelemetry.
Langfuse, integración con LangChain por retrollamadas: https://langfuse.com/integrations/frameworks/langchain.
langchain-ai/helm, chart del servidor: https://github.com/langchain-ai/helm.
Boletín Oficial del Estado, Real Decreto 311/2022, Esquema Nacional de Seguridad: https://www.boe.es/buscar/act.php?id=BOE-A-2022-7191.

Making the model prefer our tools: the priority that exists in no layer, the two primitives that do, and the `_meta` that crosses the gateway

Tue, 15 Sep 2026 03:30:00 +0200

A follow-up to LiteLLM’s MCP gateway, which treated the tool catalogue as cost and as surface. This one treats the next question, the one that arrives in the meeting afterwards: so then, how do I get the model to use mine. Verified against LiteLLM 1.102.0 and MCP specification 2026-07-28, with a test of our own run on 14 September 2026.

TL;DR

Tool priority does not exist as a concept in any layer of the stack. Not in the specification, not in the clients, not in the providers’ APIs. What exist are two distinct primitives: removing the alternative from the catalogue, and leaving it behind a search step. Everything else is persuasion, and the model ignores it when it suits.

The specification has no ranking field. The Tool object is name, title, icons, description, schemas, five annotations that are unreliable hints by their own declaration, and _meta. There is a trap I see repeated: MCP does define a numeric priority, but in another structure, the one that applies to content blocks, resources and prompts. It does not govern tool selection.

The only filtering the specification blesses is by credential. The current revision says the set of tools must not vary by connection, and that it may vary according to the authorisation presented. That is exactly what a gateway with per-key lists does.

The client is the one that decides, and only one of them has anything resembling priority. Claude Code lets you exempt a server from deferral with alwaysLoad, so that its tools load in full at start-up while the rest require a prior search. It is not called priority and it works as such.

We proved that the per-tool mark crosses the gateway. We set up an MCP server with one marked tool, served it through LiteLLM 1.102.0 and captured the wire: the _meta arrives whole and with the correct key. The chain is clean on purpose, with a comment in the code that says so.

And it is lost on exactly three routes, which are the gateway’s three catalogue-trimming modes. Proxy mode, tool search driven by key permissions, and the REST listing. In other words: the per-tool mark and LiteLLM’s catalogue trimming are incompatible. You have to choose one.

There is a library trap that makes the field disappear silently. The SDK’s Tool model declares the alias without allowing population by field name, so building it with the field name puts the dictionary in the extras and serialises it under the wrong key. There is no error. The field simply does not arrive.

The cheap lever is the description, not the name. Swapping descriptions shifts the selection distribution substantially; changing only the name has minimal and inconsistent effects. And the gateway’s overrides really do replace what is sent to the client.

Reordering the catalogue is a weak lever. The model is already attending to the correct tool eighty per cent of the time when it fails. Interventions on prompt order repair at most 23 % of the failures.

And the asymmetry you build is attack surface. Whoever controls the order controls the selection: with injection rates of 1.2 % you can take over the head of a tool ranking between 91 % and 97 % of the time.

You are here: the question that arrives after the third server

The previous article closed with a decision ladder for the catalogue: trim, rewrite, split by route, and use progressive disclosure when the catalogue grows. That ladder answers how much catalogue to show.

The question that arrives afterwards is a different one, and it is more uncomfortable. A real agent has native tools from the client hosting it, it has the in-house gateway with the house servers, and it has two or three third-party servers that somebody connected. All of them compete. And what you want is not to show less catalogue: you want that, faced with a query three tools could serve, yours wins. The audited one, the one with access control in place, the one that leaves a trace.

The answer everybody gives is to write a rule in the agent’s instruction file politely asking it to prefer the in-house server. There is even a rule published in a template marketplace that does exactly that. It is text in the prompt and the model follows it when it feels like it.

This article is what lies underneath.

The analogy: the counter with two trays

Let us go back to the switchboard operator from the previous article, the one who also hands out keys. Now the building has three key providers: the house cabinet, the maintenance contractor’s, and the cleaning service’s. All three have a key to the store room and all three open it.

The head of security wants the house one used, because it is the one with a log. And he discovers he can do three things, not one more.

He can take the other two off the counter. It always works and it annoys whoever needed them.

He can leave his own in the front tray and the other two in a drawer, so that to take one you have to ask. It works almost always and annoys nobody.

And he can change the label on his key so that it describes better when it is the right one. It works sometimes, it is the cheapest, and it is the only one that also protects against the contractor changing his label without warning.

What he cannot do is put a number from one to ten on each key and trust the operator to respect it. That number does not exist. The rest of the article is why it does not exist and what to do instead.

What the specification says: nothing

The current revision is 2026-07-28. Read against the source schema, the Tool object has name, title, icons, description, input schema, output schema, annotations and _meta. There is no field for priority, weight, rank, cost, group or tag.

The annotations are exactly five: title, and the read-only, destructive, idempotent and open-world hints. The schema itself warns that all of them are hints and that a client should not make tool-use decisions based on them when they come from untrusted servers.

The `priority` that exists and is no use

Here is the mistake I see repeated most. MCP does define a priority field, numeric between zero and one. It is in the annotations interface that applies to content blocks, to resources and to prompts, alongside the audience and the last-modified date.

It is not in Tool. It has no relation whatsoever to which tool the model picks. Anyone who finds it by searching the word in the schema and concludes that tool priority is standardised has the wrong structure.

The only precedence the specification defines for a tool is one of presentation: to display the name, first the title, then the annotations title, then the name.

The only blessed filtering is by credential

There is a new sentence in the current revision that is genuinely useful and that goes unnoticed. The set of tools must not vary by connection nor as a side effect of other requests, and it may vary according to the authorisation presented in the request.

Translated: segmenting the catalogue by key, by team or by token is the legitimate way to do it. Segmenting it by session state stopped being legitimate when the July revision removed protocol sessions.

Discovery does not discover tools

server/discover, which the current revision adds and which servers have to implement, does not return tools. It returns supported versions, capabilities, server instructions, identity and the cache freshness fields. It accepts neither filters nor a cursor.

Listing is still asking for everything and paginating. The listing request accepts only a cursor: there is no query, no filter, no limit, no groups, no tags.

And on collisions between servers, the specification explicitly and reasonedly washes its hands: clients or proxies that aggregate tools from several servers will encounter collisions and should implement a disambiguation strategy, for example prefixing. And it adds that the server name is not guaranteed to be unique and should not be used to disambiguate. There is no precedence rule between servers. There is not going to be one soon.

What there is in proposals

Nothing accepted. The official proposals index only lists those in a final state, and none of them deals with priority, groups or search. In draft there is one asking for a free-text query in the listing and a filtering capability; another on groups and tags that ended up superseded; and one carrying the word priority but which is per-tool model routing, which is another thing.

The interest group exploring the grouping of primitives declares in its own minutes that it is not going to pick a canonical pattern soon, and the documents in its repository are empty. The year’s roadmap does not mention the problem.

The layer that decides: the client

If the specification gives you nothing, what is left is whatever each client has built on its own. And here there is a clear winner.

Claude Code

Tool search is on by default, and what it does is defer: MCP tools are not loaded in full at start-up, the model sees just enough and discovers them when it searches. It requires a model that supports tool reference blocks, that is Claude Sonnet 4.5, Haiku 4.5, Opus 4.5 and later.

On top of that, the lever:

{
 "mcpServers": {
 "gateway-casa": {
 "type": "http",
 "url": "https://litellm.interna.svc/mcp",
 "alwaysLoad": true
 }
 }
}

The documentation describes it bluntly: if a server’s tools must always be visible without a search step, you set alwaysLoad to true and all of that server’s tools are loaded into context at the start of the session, regardless of the global search setting. And it recommends using it for a small number of tools that the model needs on every turn, because each tool loaded up front consumes context.

That is visibility asymmetry, and it works as priority even if it is not called that. Ours in front, the rest behind one step.

There is an operational consequence that the documentation mentions and that matters more in a gateway than in an ordinary server: setting alwaysLoad makes start-up wait for that server to return its tools, capped by the standard five-second connection timeout. A gateway that aggregates eight upstream servers and does not cache the listing, as is the case, lists live against all eight every time. If one of them is slow, the session’s start-up feels it.

There is also the fine-grained variant. An MCP server can mark individual tools as always loaded by including "anthropic/alwaysLoad": true in the tool’s _meta object, with the same effect for that tool alone. That would let you load three tools from our gateway up front and leave the other forty from the same gateway deferred. It is the lever that matters, and it is the one that motivated the test in the next section.

The rest of this client’s arsenal is pure exclusion, and it is worth knowing one detail that decides whether it works or not. A deny rule with the bare tool name removes it from the model’s context; with parentheses, it does not. The documentation puts it in a table: plain WebFetch removes the tool and the model cannot search at all, whereas WebFetch(domain:*) keeps the tool and rejects every request. The difference is not cosmetic: in the second case the model still sees the tool, still tries it and still spends the context its definition occupies.

The --tools flag is the scalpel: it accepts the empty string to disable all the built-ins, and it does not affect MCP tools. That is, --tools "" leaves the model with only what the gateway serves. It is a command-line flag and has no equivalent as a settings-file key; the functional equivalent there is deny rules.

The other clients

Client	Disable the native ones	Filter the MCP ones	Priority
Claude Code	`--tools ""`, deny by bare name	deny by pattern, subagents	`alwaysLoad`
Codex CLI	`features.shell_tool` to false	`enabled_tools` and `disabled_tools` per server	No
Gemini CLI	`tools.core` as an allowlist	`includeTools` and `excludeTools` per server	No
Copilot in VS Code	Not documented	picker and tool sets	No
Cursor	No	per-server switches	No

Two notes. Copilot cuts off at 128 tools per request and its virtualisation threshold does not go above that; its embedding-based routing is internal and has no configuration surface. And Cursor’s rules are purely indicative: they are text prepended to the context.

The API layer

If the client is in-house, the margin is wider, though narrower than it looks.

Forcing a set is not possible. In Anthropic’s case, tool choice accepts auto, any, a specific one by name, or none, and the documentation says explicitly that pointing at a set of MCP tools or at a member is not supported. In vLLM it is the same: a specific one, all of them or none. There is no way to say “any of mine from the gateway”. The proposal that would introduce grammars restricting tool names to the request’s set has been open for months and is not merged.

Filtering the catalogue is possible, in all three of the big ones. In Anthropic, with an MCP tool-set block where the default configuration disables and specific ones are enabled. In OpenAI, with the allowed tools list inside the MCP block itself, which the documentation justifies on latency and cost grounds, avoiding the model seeing unnecessary definitions. And in Gemini, the remote MCP server configuration accepts restricting which of the server’s tools the agent may call.

It is worth not confusing two things that are named almost identically in OpenAI: the allowed tools list inside the MCP block filters the catalogue, and that is what matters here; the identically named tool-choice type is another thing, it is a selection restriction, and there is evidence that it does not combine well with hosted tools, without the current documentation recording it. If anyone depends on that, let them test it against their model: it fails as a request error, so it shows up on a cold start.

And there is a 2026 lever that almost nobody is using. Anthropic has in beta a mechanism for mid-conversation tool changes, with add and remove blocks that accept referencing an entire MCP set. The reason it exists is in its documentation: the tools array sits even earlier in the chunked prefix than the system field, so editing it invalidates the cache for the whole conversation; by declaring the full set at the start and using the blocks, the array never changes and the cached prefix stays intact.

Applied to our case: you can remove a rival server’s set hot, mid-session, without paying for the whole prefix. It is not available on every model.

The test: the `_meta` crosses the gateway

The fine-grained variant of Claude Code’s lever depends on a question no documentation answers: whether a gateway that aggregates upstream servers propagates each tool’s _meta all the way to the client, or loses it along the way.

We checked.

The setup

A minimal MCP server with the Python SDK, serving two tools over HTTP, one of them with the mark in place:

types.Tool(
 **{
 "name": "always_loaded_tool",
 "description": "tool with _meta",
 "inputSchema": {"type": "object", "properties": {}},
 "_meta": {"anthropic/alwaysLoad": True, "probe": "UPSTREAM_META_MARKER"},
 }
)

In front of it, a LiteLLM from the main branch with that server declared in config.yaml, with no database, with a master key. And an MCP client querying the listing against /mcp.

The result

This is what comes out on the wire:

{
 "_meta": {"litellm.ai/server_outcomes": {"probe": {"status": "ok", "tool_count": 2}}},
 "tools": [
 {
 "name": "probe-always_loaded_tool",
 "description": "tool with _meta",
 "inputSchema": {"type": "object", "properties": {}},
 "_meta": {"anthropic/alwaysLoad": true, "probe": "UPSTREAM_META_MARKER"}
 }
 ]
}

The mark arrives whole and with the correct key. The client parses it without trouble.

Tracing the code, the chain is clean on purpose. LiteLLM keeps the SDK objects as they are when it lists against the upstream, instead of rebuilding them. The name prefixing does a deep copy and mutates only the name, with a comment in the code declaring the intention to preserve every field including the _meta by avoiding mutation. The two permission filters are list comprehensions that reuse the same objects. And the name and description overrides mutate in place, so they do not lose anything either. Somebody thought about this.

One operational detail: the name arrives prefixed with the server alias, governed by the configurable separator and the short prefix mode.

The three routes where it is lost

And here is the finding that changes the previous article’s recommendation.

Proxy mode returns only its three fixed tools, with no _meta and with no upstream tool at all. Verified against the wire as well.

Tool search activated by key permissions does the same with four virtual tools.

And the REST listing builds its response object by hand and discards the field: the tool that did carry a mark comes out with the field null. The class inherits from the type that has the field; it simply is not filled in. The fix fits on one line.

Those three routes are, exactly, the three ways LiteLLM has of trimming the catalogue. The operational conclusion is uncomfortable and it is worth saying plainly: either you use the gateway’s catalogue trimming, or you use the client’s per-tool mark. They cannot be combined. The previous article recommended proxy mode as the fifth rung of the ladder; with this in hand, that rung and the fine-grained mark are mutually exclusive.

The library trap

This deserves its own section because it takes out anyone writing a server or a proxy, and it gives no warning.

The SDK’s tool model declares the field with an alias, and does not enable populating it by name. The consequences, both checked by running them:

Building with the field name populates nothing. The dictionary slips into the model’s extras and comes out serialised under the wrong key, without the leading underscore, which is a key no client interprets.

And dumping and revalidating without asking for aliases loses the field, for the same reason.

There is no exception, no warning in the log, nothing. The field disappears. If someone marks their tools and does not see them marked in the client, this is the first place to look, before the gateway.

What we have not tested

For honesty’s sake, and because it is the link that remains: we have shown that the mark survives the gateway. We have not shown that the client acts on it when the tool arrives with the name prefixed by the gateway. Claude Code’s documentation describes the per-tool mark in a single sentence and says nothing about aggregating servers or about prefixes.

It is a half-hour test for anyone who has the setup in front of them: two tools from the same gateway, one marked, and look at which one appears loaded at the start of the turn. If someone does it before I do, I am interested in the result.

What LiteLLM can and cannot do

With the above, the gateway’s inventory of levers comes out like this.

Lever	Serves external MCP clients	Preserves the `_meta`
`allowed_tools` and `disallowed_tools` per server	Yes	Yes
Name and description overrides	Yes	Yes
Tool sets per route	Yes	Yes
Proxy mode and virtual tools	Yes	No
Search by key permissions	Yes	No
REST listing	Yes	No
Semantic filter	No	Not applicable

Two clarifications on the previous article’s table, now that the code has been read again.

The allowed and disallowed lists are applied in the listing, not only on the call. They are the blunt, effective instrument, and they do not break the cache because they are static.

And the overrides have a limit that was not accounted for: they are applied in the MCP protocol listing, but they are skipped in proxy mode and not applied on the REST route. Anyone rewriting descriptions and also turning on proxy mode is not serving what they think.

What the gateway cannot do is inject _meta. There is no setting equivalent to the ones for name and description. The mark has to be put there by the origin server, which for in-house servers is trivial because we write them. To mark third-party tools you would have to patch, and it is small: one field in the type and three lines in the overrides function if configuring it by file is enough.

There is also an ordering lever that did not appear in the previous article. In the proxy’s search mode there is a core tools setting that puts them at the front of the ranking and that also does not count against the results cap. It is the closest thing to a declarative priority in the whole stack, and it lives inside the one mode that discards the _meta.

And a nuance about proxy mode’s results cap. The client cannot ask for more than five, that is true, but the operator can raise it by configuration. The previous article implied it was immovable.

The cheap lever: the description, not the name

If you cannot exclude, what is left is to bias. And bias comes in through the description.

The work that measures selection bias between functionally equivalent tools puts the combined bias of the evaluated models between 0.25 and 0.38, that is, you would have to redistribute between 25 % and 38 % of the probability mass for equivalent tools to be picked equally. On that basis, swapping two tools’ descriptions shifts selection substantially, whereas perturbations that touch only the name produce smaller and more inconsistent effects.

Another piece of work measures that a single pass of automatic description rewriting improves the metric over a large corpus, almost as much as iterative refinement, and with two orders of magnitude less time. And it adds a warning worth retaining: descriptions tuned against a fixed set of candidates do not generalise to the dynamically retrieved set. You have to optimise for the regime you serve in.

Applied: the gateway’s description override is at once token trimming, selection bias and mitigation of description change after approval. It is the lever with the best effort-to-effect ratio in the whole article, and it is static, so it does not touch the prompt cache.

Name prefixing, by contrast, costs tokens and probably does not change which tool the model goes to.

What does not work

Reordering the catalogue. A June paper measures that, when the model fails, it was already attending to the correct tool eighty per cent of the time, well above chance. The bottleneck is not at the input but in the late layers. Interventions on prompt order recover at most 23 % of the failures, against the 59 % to 91 % of those acting on the final readout. Reordering is cheap and that is why it gets recommended a lot; it is also weak.

Rules and instructions. They are text. They help and they do not decide.

Your own per-request top-K filters. This is covered in the previous article and it still holds: four out of five measured strategies perform worse than not filtering, and on top of that rewriting the tool block on every turn invalidates the entire cached prefix.

The ladder, updated

Exclude in the client. It is the only deterministic thing. --tools "" or deny by bare name for the competing native tools.
Mark your own as always loaded. At server level with alwaysLoad, or per tool with the mark in the _meta if you control the origin server. Counting on start-up waiting for the server.
Trim with static lists per server and per key, and split a catalogue by agent profile across routes.
Rewrite the descriptions with the gateway’s overrides.
Remove hot the rival sets, if the client is in-house and the model supports it.
Choose: the gateway’s catalogue trimming, or the per-tool mark. Not both.
Do not: reorder, rename, or trust prompt rules.

The risk that has to be declared

The asymmetry you build is also a surface. Whoever controls which tools head a ranking controls the selection, and that can be attacked: there is work measuring that by injecting adversarial tools at rates of 1.2 % you take over the head of the ranking between 91 % and 97 % of the time.

In a system under ENS (Esquema Nacional de Seguridad, Spain’s national security framework) this fits under change management, not access control: the set of tools a model sees and the order in which it sees them are security configuration, and today nobody versions them. Neither does the specification fix a hash of the description or of the schema, nor does the gateway. The specific Annex II codes are worth checking against the current text before taking them into a compliance document; the approach is developed in the MCP gateway article.

There is a precedent within the ecosystem itself that points the way. The MCP skills extension, which is in a final state, requires a manifest with a per-file hash, mandatory verification before use, and approval tied to the set of files and their digests, so that any change revokes the approval. And it obliges hosts to prevent two skills with the same name from silently replacing one another. It is exactly the pattern missing for tools. Anyone who needs it today has to implement it in their gateway.

Checklist

Decide the strategy before touching anything: catalogue trimming in the gateway, or the per-tool mark in the client. They are mutually exclusive.
If you pick the mark, put it in the origin server, because the gateway cannot inject it.
Build the tool object with the alias key, never with the field name, or the field disappears without warning.
Do not revalidate tool objects from a dump without asking for aliases.
Count on marking a server as always loaded making session start-up wait for that server, and on a gateway listing live against all of its upstreams.
Exclude in the client the native tools that compete, by bare name and not with a pattern in parentheses.
Rewrite the descriptions of your own tools, and do not waste time renaming.
Do not turn on proxy mode if you depend on the description overrides, because they are not applied there.
Version the exposed tool set and its order as security configuration, with a hash of the description and the schema.
If you use the gateway’s REST route for anything, know that it discards the _meta.

Traps and things that are not what they look like

MCP’s priority exists, and it is not about tools. It lives in the annotations for content, resources and prompts.
Tool annotations are hints and the schema itself says they are not reliable from untrusted servers.
The specification gives no precedence rule between servers, and it says besides that the server name is no use for disambiguating.
The tool set may indeed vary by authorisation, and that is the only blessed segmentation.
server/discover does not return tools.
A deny rule with parentheses does not remove the tool from the context, it only rejects the calls.
--tools is a command-line flag, with no equivalent in the settings file.
Marking a server as always loaded delays start-up by up to five seconds per server.
Tool(meta=...) does not populate the field and serialises it under the wrong key.
Proxy mode, search by permissions and the REST listing discard the _meta.
Description overrides are skipped in proxy mode and are not applied on REST.
Proxy mode’s cap of five results can be raised by the operator, even though the client cannot ask for more.
The core tools of the proxy’s ranking are the only declarative priority in the stack, and they live in the mode that loses the _meta.
You cannot force “any of mine from the gateway” in any API.
In OpenAI, the allowed list inside the MCP block and the identically named tool-choice type are different things.
Gemini does accept a per-tool allowlist on its remote MCP servers.
Reordering the catalogue repairs at most 23 % of the failures.

Closing

The starting question had a trap in it, and the trap is the word. When someone asks how you prioritise an MCP server, they are assuming a dial exists somewhere. It does not, and I have spent the whole article showing the places where it is not.

What there is instead is poorer and more manageable: you remove what competes, you leave your own in front, and you write the description better. Three things, none of them elegant, all three effective in that order.

What does seem to me worth taking away is the shape of the problem. The tool catalogue a model sees is security configuration in the full sense, because it determines what the agent can do and which system it is going to talk to. And today it is managed like a list of connections: things get added, it is not versioned, it is not signed, and nobody detects a change. The ecosystem has already solved that problem once, for skills, with hashes and tied approval. For tools, not yet.

In the meantime, the asymmetry has to be built by hand, server by server, and knowing that it rests on the _meta of an object that a library can empty without saying anything.

Sources

Model Context Protocol, herramientas en la revisión 2026-07-28: https://modelcontextprotocol.io/specification/2026-07-28/server/tools.
Model Context Protocol, server/discover: https://modelcontextprotocol.io/specification/2026-07-28/server/discover.
Model Context Protocol, paginación: https://modelcontextprotocol.io/specification/2026-07-28/server/utilities/pagination.
Model Context Protocol, caché de listados: https://modelcontextprotocol.io/specification/2026-07-28/server/utilities/caching.
Model Context Protocol, esquema fuente de la revisión: https://github.com/modelcontextprotocol/modelcontextprotocol/blob/main/schema/2026-07-28/schema.ts.
Model Context Protocol, extensión de habilidades: https://modelcontextprotocol.io/extensions/skills/overview.
Model Context Protocol, grupo de interés sobre agrupación de primitivas: https://modelcontextprotocol.io/community/interest-groups/primitive-grouping.
Anthropic, Claude Code y MCP (alwaysLoad, búsqueda de herramientas): https://code.claude.com/docs/en/mcp.
Anthropic, Claude Code, referencia de línea de comandos (--tools): https://code.claude.com/docs/en/cli-reference.
Anthropic, Claude Code, permisos (denegación por nombre pelado): https://code.claude.com/docs/en/permissions.
Anthropic, conector MCP y conjuntos de herramientas: https://platform.claude.com/docs/en/agents-and-tools/mcp-connector.
Anthropic, cambios de herramientas a mitad de conversación: https://platform.claude.com/docs/en/build-with-claude/mid-conversation-system-messages.
Anthropic, búsqueda de herramientas: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool.
OpenAI, MCP y conectores (lista de herramientas permitidas): https://developers.openai.com/api/docs/guides/tools-connectors-mcp.
OpenAI, búsqueda de herramientas y caché: https://developers.openai.com/api/docs/guides/tools-tool-search.
Google, llamada a funciones en la Interactions API: https://ai.google.dev/gemini-api/docs/function-calling.
vLLM, llamada a herramientas y decodificación restringida: https://docs.vllm.ai/en/latest/features/tool_calling.html.
vLLM, propuesta de decodificación guiada por regiones y gramáticas de herramienta: https://github.com/vllm-project/vllm/issues/39848.
LiteLLM, código del gateway MCP: litellm/proxy/_experimental/mcp_server/server.py, mcp_server_manager.py, rest_endpoints.py, tool_search.py, litellm/experimental_mcp_client/tools.py: https://github.com/BerriAI/litellm.
SDK de Python de MCP, modelo Tool y serialización por alias: https://github.com/modelcontextprotocol/python-sdk.
BiasBusters, sesgo de selección entre herramientas equivalentes y peso de la descripción frente al nombre: https://arxiv.org/html/2510.00307v2.
Looking Is Not Picking, atención frente a lectura final y techo de las intervenciones sobre el prompt: https://arxiv.org/html/2606.16364v1.
A Single Rewrite Suffices, reescritura de descripciones y generalización al conjunto recuperado: https://arxiv.org/html/2606.30775.
ToolFlood, saturación adversaria de la capa de recuperación: https://arxiv.org/html/2603.13950.

Ia on lo0 — Tech Blog

Real ClickHouse capacity in Langfuse v4: 2,782 bytes per observation, and why almost half of that is not data

TL;DR

You are here: the sizing you cannot do by reading

The analogy: the archive and the index card

The benchmark

What one observation takes up

Where the bytes go

The split by observation type

The indexes, which are almost half

The materialized view as an amplifier

What happens on deletion

Projection

What this benchmark does not measure

Checklist

Traps and things that are not what they look like

Closing

The series: the ten articles

See also

Sources

deepagents on a cluster of your own: the SDK is free, the server is not, and what you have to build in between

TL;DR

You are here: the harness arrives after the model

The analogy: the scaffolding and the crane

What deepagents actually is

What is no longer there

Where state lives

The backends

What grows in the database

The store, which does have cleanup

The isolation that does not exist

The model, and the trap you need to know about

The tools, and the MCP that is not where it seems

The subagents

Code execution

Observability, and the package it goes through

The server: here is where the free part ends

What you have to build

How it fits with activity logging

Checklist

Traps and things that are not what they look like

Closing

See also

Sources

Making the model prefer our tools: the priority that exists in no layer, the two primitives that do, and the `_meta` that crosses the gateway

TL;DR

You are here: the question that arrives after the third server

The analogy: the counter with two trays

What the specification says: nothing

The priority that exists and is no use

The only blessed filtering is by credential

Discovery does not discover tools

What there is in proposals

The layer that decides: the client

Claude Code

The other clients

The API layer

The test: the _meta crosses the gateway

The setup

The result

The three routes where it is lost

The library trap

What we have not tested

What LiteLLM can and cannot do

The cheap lever: the description, not the name

What does not work

The ladder, updated

The risk that has to be declared

Checklist

Traps and things that are not what they look like

Closing

See also

Sources

The `priority` that exists and is no use

The test: the `_meta` crosses the gateway