Networkpolicy on lo0 — Tech Blog

Hardening and secrets in the sovereign LLM stack: defence in depth

Thu, 11 Jun 2026 11:00:00 +0000

Part of the operational series on squeezing a generic on-premise 4×H100 SXM 80GB LLM cluster. The sibling pieces: the document ingestion from PDF to indexed chunk that fills the vector database, the embeddings and rerankers service with TEI in production that feeds it, and, the most directly coupled to this post, the GitOps of the inference stack with Flux, because GitOps and secrets share a chicken-and-egg problem we solve here. Assembling all of this into a conversational assistant (LibreChat + LiteLLM + RAG) is covered by another post in the series, still in draft.

TL;DR

A sovereign LLM inference stack is, at minimum, six services: a gateway (LiteLLM or equivalent), an inference engine (vLLM), an embeddings/rerankers service (TEI), a vector database, a state database (conversations, users) and a front end (LibreChat). Six services are six attack surfaces, and the assistant “works” as soon as the gateway returns tokens, long before it is hardened. This post walks through defence in depth layer by layer:

Secrets: never in the clear in git. sealed-secrets (asymmetric encryption, controller in the cluster) vs SOPS/age (file encryption, simple, pure GitOps) vs External Secrets Operator + Vault (dynamic secrets, native rotation). GitOps’s chicken-and-egg problem.
Network: default-deny NetworkPolicy + explicit allow; L3/L4 with the standard NetworkPolicy, L7 and DNS-based egress with Cilium (eBPF); egress control so the data does not leave the perimeter; internal mTLS.
Pod security: runAsNonRoot, readOnlyRootFilesystem, dropped capabilities, seccomp RuntimeDefault, restricted Pod Security Standards, nothing privileged.
Supply chain: pin by digest, scan with Trivy, sign with cosign, admission control that rejects anything unsigned.
AuthN/Z: LiteLLM virtual keys, OIDC/LDAP in the front end, Kubernetes RBAC with least privilege.
Runtime: detection and, optionally, enforcement with Tetragon.
Data at rest: storage encryption, credentials for the vector database and the state database.

The thesis: hardening reduces the blast radius, it does not eliminate it. That is why you prioritise by impact, and the first job on a 4×H100 is the gateway↔vector store pair and egress-deny.

The analogy: the sovereign office

Picture an office holding sensitive documentation: the corpus that feeds the RAG, the users’ conversations, the engine’s credentials. It is not enough for the street door to lock. Real security is defence in depth: several layers, each assuming the previous one can fail.

The mapping is direct:

The building’s perimeter is egress control: making sure a leaked secret or a compromised process cannot exfiltrate the corpus to an external server. It is the most underrated layer and the first one I put in place in a sovereign deployment.
Access control at every interior door is the default-deny NetworkPolicy: the front end does not talk to the vector database directly because it has no reason to; only the strictly necessary pairs are open.
The fire partitions are pod isolation (restricted) and namespaces: if one service catches fire, the fire does not jump to the one next door.
The doorman who checks credentials is the admission control that verifies each image’s cosign signature before letting it into the cluster.
The cameras are Tetragon/Falco: they record what each process did and detect (or kill) anything anomalous.
The safe with a rotated key is the secrets: encrypted at rest, out of git, and rotated so a stolen key expires.

No layer is sufficient on its own. The building is secure because crossing them all is expensive.

Layer 1 — Secrets: GitOps’s chicken-and-egg problem

The stack is deployed by GitOps: the sibling piece on Flux reconciles the state declared in git against the cluster. That is excellent for manifests, but there is a foundational problem: the inference engine needs the Hugging Face token to download the model, the state database needs its password, the gateway needs its master key. Those secrets cannot go into git in the clear. Anyone with read access to the repo, and in an organisation with generous read that is a lot of people, plus every repo backup, plus every fork, would read them.

The chicken and the egg: GitOps wants all state to be in git, but secrets cannot be in git in the clear. The solution comes in three families:

Family A — sealed-secrets (asymmetric encryption, controller in the cluster)

Bitnami Sealed Secrets is a controller running in the cluster plus a client CLI, kubeseal. It uses asymmetric cryptography: there is a key pair. The public one is held by developers and is used to encrypt; the private one lives only in the cluster’s controller and is used to decrypt. The flow:

The developer takes a normal Secret and encrypts it with kubeseal, which obtains the controller’s public key. Result: a SealedSecret resource.
That SealedSecret is committed to git in the clear — it is encrypted, it is not readable.
The controller in the cluster detects it, decrypts it with its private key and creates the real Kubernetes Secret in the target namespace.

The key property: because the private key never leaves the cluster, neither the developer nor anyone with access to git can decrypt. And the encryption includes the namespace name: a SealedSecret sealed for inferencia cannot be moved to front and decrypted there — it behaves as if each namespace had its own key. The controller also manages rotation of the sealing keys, labelling them as active or compromised.

Advantage: it fits pure GitOps perfectly, the encrypted secret lives with the rest of the state. Limitation: the secret, once decrypted, ends up as a normal Kubernetes Secret — it sits in etcd, and etcd has to be encrypted separately (we will see that in layer 7).

Family B — SOPS + age (file encryption, simple)

SOPS encrypts the whole YAML/JSON file (or only its values) and leaves it in git encrypted, decrypting it only at deployment time. It supports cloud KMS (AWS/GCP/Azure), PGP and, the relevant one for on-premise sovereignty, age, a simple serverless offline encryption scheme. The GitOps operator (Flux brings native SOPS integration) decrypts on reconciliation using the age key stored in the cluster.

Advantage: simple, scriptable, with no server to maintain; the operational weight falls on safeguarding the age key. It is the usual recommendation for small teams or for getting started. Limitation: rotation is manual (re-encrypt everything with the new key) and there are no dynamic secrets.

Family C — External Secrets Operator + Vault (dynamic secrets, native rotation)

The External Secrets Operator (ESO) keeps no secrets in git at all. All that goes into git is an ExternalSecret: a reference saying “the password field comes from path secret/data/vectordb in such-and-such a SecretStore”. The operator reads from an external store, typically HashiCorp Vault, and synchronises the Kubernetes Secret. Vault keeps the secrets in its own encrypted backend and serves them over an authenticated API, so that they never reside in git and, with its dynamic engines, it can issue short-lived database credentials that expire on their own.

Advantage: native rotation, dynamic secrets, centralised auditing, a single point of governance. Limitation: you have to operate Vault (sealing/unsealing, high availability, access policy), which is real work.

How to choose, and rotation

Criterion	sealed-secrets	SOPS + age	ESO + Vault
Secret in git	encrypted	encrypted	reference only
Extra server	lightweight controller	none	Vault (heavy)
Rotation	automatic sealing keys	manual (re-encrypt)	native / dynamic
Dynamic (short-lived) secrets	no	no	yes
Fit with pure GitOps	excellent	excellent	good (references)
Operational curve	low	very low	high

For a sovereign 4×H100 getting started, SOPS/age or sealed-secrets cover 90% at minimal cost. When the system grows and requirements for frequent rotation and auditing appear, typical in ENS High Category scenarios, you migrate to ESO + Vault. The rotation rule: a secret that never rotates is a secret that, once leaked, is leaked forever. Rotation does not prevent the leak; it bounds its window of validity. We will come back to that when we talk about blast radius.

Layer 2 — Network: default-deny, egress and the arithmetic of the surface

The arithmetic of communication pairs

Without a network policy, in Kubernetes every pod can talk to every pod. That is the default posture, and it is the worst one for a sovereign system. With $N$ services, the number of possible ordered communication pairs (who-can-call-whom) is:

$$P_{\text{abierto}} = N \cdot (N-1) \approx N^2$$

With our $N = 8$ components (gateway, vLLM, TEI, vector database, state database, front end, plus the secrets controller and the observability one), that is $8 \cdot 7 = 56$ possible directed pairs. Fifty-six paths along which a compromised service could pivot laterally.

Now we apply a default-deny NetworkPolicy: nothing talks to anything except what is explicitly allowed. The real whitelist of an LLM stack is small. The strictly necessary pairs:

front end → gateway
gateway → vLLM
gateway → TEI
gateway → vector database
gateway → state database
front end → state database (sessions/users)

That is $E = 6$ edges. The communication surface falls from $56$ to $6$:

$$\frac{E}{P_{\text{abierto}}} = \frac{6}{56} \approx 0.107$$

Almost 89% of the possible paths are closed. The front end can no longer touch vLLM or TEI or the vector database directly; if somebody compromises the front end, they have no network route to the corpus. This is the difference between $\sim N^2$ and a whitelist $E \ll N^2$.

Standard L3/L4, L7 and egress with Cilium

Kubernetes’ native NetworkPolicy operates at L3/L4: it selects pods by label and allows/denies by port and protocol. That covers the bulk of the whitelist. But it has limits: it does not understand DNS or HTTP. This is where Cilium comes in, applying policies in the kernel via eBPF and extending them with CiliumNetworkPolicy:

Egress by FQDN/DNS: instead of pinning IPs (which change), allow egress only to huggingface.co so vLLM can download the model and block the rest. Critical for egress control.
L7: allow only certain HTTP methods/paths between the gateway and an internal service.
Enforcement modes: in the default mode, an endpoint with no policy selecting it has everything open; in always mode, everything is denied until a policy explicitly opens it. For a sovereign system, always + default-deny is the target posture.

Egress control is the sovereign piece par excellence. An internal service being unable to open connections to the internet except to a handful of explicit destinations means that, even if vLLM or the gateway is compromised, the corpus cannot leave the perimeter. The RAG is the asset of value; egress-deny is what stops it leaking out.

Internal mTLS

The NetworkPolicy says who can talk to whom, but it neither encrypts nor authenticates east-west traffic. For that, mTLS (mutual TLS: client and server authenticate each other). Cilium offers native mutual authentication; alternatives such as Linkerd or Istio give you a full service mesh. For a six-service stack, Istio’s full mesh is usually over-engineering; Cilium’s mutual authentication or Linkerd (lighter) is proportionate. The effect: an attacker on the cluster network cannot impersonate the gateway or eavesdrop on internal traffic in the clear.

Layer 3 — Pod security: restricted, and nothing privileged

The Pod Security Standards define three profiles: privileged (no restrictions), baseline (the reasonable minimum) and restricted (hardened). The target for the whole LLM stack is restricted, applied via Pod Security Admission with a namespace label in enforce mode. The restricted profile requires:

runAsNonRoot: true and a runAsUser other than 0. No container runs as root.
readOnlyRootFilesystem: true. The root filesystem is read-only; anything that needs to write goes to an emptyDir or an explicit volume. An attacker cannot leave persistent binaries in the container.
drop: ["ALL"] of Linux capabilities. No capabilities that are not needed.
seccomp RuntimeDefault, which applies the runtime’s profile and blocks dangerous syscalls.
allowPrivilegeEscalation: false, nothing privileged, no host namespaces.

The nuance with GPUs: pods using an H100 load NVIDIA’s device plugin, which historically tempted people to relax the securityContext. You do not need to run the inference container as privileged in order to use the GPU; device access is managed by the plugin, and the vLLM pod can and should run restricted. This is a fire partition: if vLLM is compromised, the attacker has a non-root process, with no capabilities, a read-only FS and trimmed syscalls — a terrible starting point for escalation.

Layer 4 — Image supply chain

The container image is third-party code running with access to your data. Three composed controls:

Pin by digest, not by tag. vllm/vllm-openai:latest is a moving target; @sha256:... is immutable. Pinning the digest guarantees that what was deployed is exactly what was audited.
Scanning with Trivy. Before promoting an image, Trivy enumerates its CVEs and composes the SBOM (component inventory). The pipeline fails if there are unmitigated critical vulnerabilities.
Signing with cosign and verification at admission. The Sigstore project (cosign for signing, Fulcio as a CA for ephemeral certificates via OIDC, Rekor as an immutable transparency log) allows the image to be signed. And Sigstore’s policy-controller, or Kyverno, is an admission controller that verifies the signature before admitting the pod: an image without a valid signature does not enter the cluster.

This is the doorman from the analogy: he checks each image’s credential (signature) at the door. A poisoned image uploaded to a registry, or a hijacked tag, stays outside because it carries no signature from the trusted issuer.

Layer 5 — AuthN/Z: virtual keys, OIDC and minimal RBAC

Three identity control points, from the outside in:

Front end (OIDC/LDAP). The human user authenticates against the corporate identity provider. The front end does not invent its own user system; it delegates to OIDC. This gives SSO, MFA and centralised revocation.
Gateway (LiteLLM virtual keys). The gateway issues virtual keys: each team, application or user has its own key with a budget, a rate limit and permitted models. The gateway’s master key, and the real API keys towards the engine, are never seen by the client; clients only handle their virtual key, revocable individually. If a virtual key leaks, that one and only that one is revoked.
Kubernetes RBAC with least privilege. Each service’s ServiceAccount has exactly the permissions it needs. The vLLM pod does not need to list Secret objects in other namespaces or create pods. Restrictive RBAC means a stolen service account token opens very little.

Layer 6 — Runtime: the cameras with Tetragon

The previous layers are preventive. What is missing is detection: what if something, despite everything, runs where it should not? Tetragon is observable security and runtime enforcement over eBPF, Kubernetes-aware. It hooks kernel events — process_exec, tcp_connect, security_file_open — with typical overhead below 1%, and it can move from observing (recording the event) to enforcement (killing the process or cutting the connection in the kernel, Sigkill).

The operational rule, which I set out in detail in the bubblewrap + Tetragon runbook: adopt first, block later. First you deploy in observation mode to build a baseline of the stack’s normal behaviour — which binaries vLLM executes, what the gateway connects to — without false positives. Only afterwards do you promote the clear rules to enforcement: kill any process that tries to read secret paths, or cut every tcp_connect to destinations outside the egress whitelist. Falco is the detection-only alternative over eBPF; Tetragon adds enforcement. These cameras also produce the audit evidence: what each service ran and which attempt was blocked.

Layer 7 — Data at rest

The last thing we protect is data standing still:

Storage encryption. The persistent volumes, where the vector database and the state database live, on top of disk encryption (LUKS/dm-crypt) or Ceph-level encryption. A disk stolen from the datacenter does not reveal the corpus.
etcd encryption. Remember that Kubernetes Secret objects, once decrypted by sealed-secrets or ESO, are normal objects in etcd. You have to enable etcd encryption at rest, or the secret sits in the clear in the control plane.
Database credentials. The vector database’s password and the state database’s are first-class secrets (layer 1), never embedded in the manifest or in environment variables in the clear in git.

Exposure table: service × port × external egress?

This table is the input for writing the NetworkPolicy resources. The egress column is the one that decides what leaves the perimeter.

Service	Internal port	Permitted callers	External egress?
front end	3080	(human ingress via OIDC)	No
gateway (LiteLLM)	4000	front end	No
vLLM	8000	gateway	Only `huggingface.co` for the initial download; zero in operation
TEI (embeddings/reranker)	8080	gateway	Only model download; zero in operation
vector database	6333	gateway	No
state database	5432	gateway, front end	No
secrets controller	—	(control plane)	No
observability	9090	internal scraping	No

The reading: no service needs external egress in normal operation. vLLM and TEI need it once to pull the model, and that permission can be temporary or restricted by FQDN to huggingface.co. Everything else is total egress-deny. If a service starts attempting outbound connections this table does not contemplate, Tetragon records it and, in enforcement, cuts it.

Blast radius: a leaked secret, with and without defence

The blast radius measures how much damage a compromise does. Let us model it for the realistic worst case: the state database’s credential leaks.

Without hardening (flat network, unrotated secret, no egress-deny, no detection):

The secret gives access to the state database from any pod (flat network → 56 open pairs).
The secret does not rotate → it is valid indefinitely; the exploitation window is $\infty$ until somebody notices.
Without egress-deny, the attacker dumps the whole database to an external server.
Without Tetragon, nobody finds out until the public incident.
Blast radius: the entire state database, exfiltrated, undetected, for an indefinite time.

With hardening (default-deny, ESO + Vault with dynamic credentials, egress-deny, Tetragon in enforcement):

The secret is only usable from the pod that has a network route to the state database (1-2 pairs, not 56).
With Vault’s dynamic credentials, the secret expires — say within a window $T$ of hours; past $T$, it is worthless.
Egress-deny prevents the dump to the internet: the attacker can read, but cannot take anything out.
Tetragon records the anomalous access and, in enforcement, kills the process attempting the outbound connection.

The qualitative reduction is enormous, but let us be quantitative about the window. If a static secret is valid forever and a rotated one with period $T$ is valid for at most $T$, and compromises arrive at rate $\lambda$, the expected number of live and exploitable secrets at any given instant goes from growing without bound to being capped at $\lambda \cdot T$. With rotation every 24 h ($T = 1$ day) against “never”, a given secret’s window falls from months to a day: a reduction of one to two orders of magnitude in temporal exposure. Combined with the reduction in network pairs ($56 \to \sim 2$, a factor of $\sim 28\times$) and egress-deny (from exfiltration possible to impossible by the direct route), the blast radius is drastically reduced.

But let us be honest: it does not reach zero. The attacker with the credential can read the state database during the window $T$ from the compromised pod. Hardening turned “indefinite, silent catastrophe” into “a bounded, detected incident with no exfiltration by the direct route”. That is exactly what defence in depth promises: not invulnerability, but that the cost of a full compromise is high and its radius small.

The ENS / NIS2 angle

These layers are not voluntary hygiene: they implement specific controls. The mapping is developed in detail in the technical controls ENS × ISO 42001 × EU AI Act post; here, the actionable summary:

Hardening layer	ENS measure (RD 311/2022)	NIS2 / framework link
Encrypted secrets + rotation	`op.exp.11` (cryptographic keys), `mp.info.3` (encryption)	Credential management; in High Cat., HSM
Default-deny NetworkPolicy + segmentation	`mp.com.1` (perimeter), `mp.com.4` (flow separation)	NIS2 art. 21: network security measures
Egress control	`mp.com.1` + `op.mon.1`	Exfiltration prevention
Internal mTLS	`mp.com.2-3` (confidentiality/integrity in transit)	Mandatory TLS
Restricted Pod Security	`op.exp.2` (hardened configuration)	Configuration hardening (CIS)
Trivy + cosign + admission	`op.exp.6` (malicious code), `op.ext.3` (supply chain)	NIS2 supply chain
OIDC + virtual keys + RBAC	`op.acc.1-2-5` (identification, access, authentication)	MFA in High Cat.
Tetragon runtime	`op.mon.1` (intrusion detection)	Monitoring and response
Encryption at rest + etcd	`mp.info.3` (encryption), `mp.si` (media)	Data at rest

The honest note for auditing: hardening reduces risk, it does not eliminate it, and maturity is demonstrated by prioritising by impact. A competent auditor does not want to see nine half-finished layers; they want to see that egress-deny on the critical asset (the corpus) and secrets management are solid before mTLS is perfect between low-sensitivity services. For the management and governance context, see also ISO/IEC 42001 as an AIMS and the EU AI Act mapping onto the architecture.

Applied to the generic 4×H100 cluster

In a real deployment on 4×H100 SXM 80GB, you do not harden the nine layers at once. The order, prioritised by impact:

Egress-deny on vLLM, TEI and the vector database, first. It is the barrier that stops the corpus leaving the perimeter, and it goes in as soon as the services start. Allow egress by FQDN to huggingface.co only during the model’s initial download; afterwards, zero. It is the layer with the highest return per hour invested.
Secrets for the inference engine and the databases, out of git. The Hugging Face token vLLM uses to download the model, the vector database’s and the state database’s passwords, the gateway’s master key: into sealed-secrets or, if there is already a rotation requirement, into ESO + Vault. Never in values.yaml in the clear.
Default-deny NetworkPolicy + the 6-edge whitelist. The gateway and the vector database are the most exposed services — the gateway because it receives all the traffic, the vector database because it holds the embedded RAG. Closing everything that is not on the whitelist cuts lateral pivoting from $\sim N^2$ to the 6 real edges.
restricted Pod Security in the inference namespace, including the vLLM pod with a GPU (it needs no privileges to use the H100).
cosign + admission so only signed images get in; Trivy in the GitOps pipeline of Flux.
Tetragon in observation, baseline, and then enforcement over the egress of the sensitive pods.
Encryption at rest for the databases’ volumes and for etcd.

The gateway and the vector database are the two I harden first inside the whitelist: the gateway for being the exposed face, the vector database for holding the asset that egress-deny protects. The rest is built on top, layer by layer, always assuming the previous one can fail.

What hardening does NOT solve

To close honestly, what these layers do not cover and what needs other pieces:

Prompt-level attacks (jailbreak, indirect injection via the RAG corpus): that is the job of guardrails and LLM Guard, not of NetworkPolicy.
Agents with legitimate permissions doing something harmful: the runtime isolation from the agent isolation post bounds what an agent can touch, but a granted permission is a usable permission.
The human factor: a well-managed secret shared over Slack is still leaked. Rotation bounds the window, it does not eliminate the mistake.
Zero-day vulnerabilities in the components themselves: Trivy detects what is known; the unknown gets through until the CVE is published.

Hardening is a multiplier on the cost of attacking, not an impassable wall. Its value lies in making an individual compromise bounded, detected and with no way out — and that, for a sovereign system holding sensitive data, is exactly the goal.

Runbook: caging the AI agent — bubblewrap on the client, Tetragon on the cluster

Tue, 09 Jun 2026 17:00:00 +0200

Operational companion to The contractor with the master key. That post explains the why and the where: the threat model, the five isolation families, which domain uses each. This one is the how, with commands. If you have not read it, read it first: here I take for granted what the blast radius is, why bwrap runs without root and what Tetragon watches. The procedure comes in two independent tracks, client and cluster, because, as the sibling post argues, the control extrapolates but the primitive gets rewritten.

TL;DR

Two reproducible procedures. Client (workstation): install ai-jail (it wraps bubblewrap), generate the per-project .ai-jail, audit with --dry-run, fix the allowlists with --bootstrap, use --lockdown for anything you do not trust, and leave the agent without git push permission. Cluster (RKE2 with Cilium + Tetragon): put the pod baseline in place (unprivileged securityContext, seccomp: RuntimeDefault, default-deny NetworkPolicy), put the untrusted agent’s pod in a microVM with runtimeClassName: kata, and deploy the Tetragon TracingPolicy files in two phases: observe with action: Post to raise the baseline, then promote to action: Sigkill on tcp_connect (egress) and security_file_open (secret paths). The golden rule of the Tetragon phase: adopt first, block later; never put a Sigkill into production without having seen the events in observation mode first.

The flow of the two tracks

Track A — Client (the developer’s workstation)

A0 — Install ai-jail and bubblewrap

ai-jail wraps the sandbox; on Linux it needs bubblewrap separately, on macOS it needs no extra dependency.

# ai-jail (macOS and Linux)
brew tap akitaonrails/tap && brew install ai-jail
# or, with cargo:
cargo install ai-jail
# or, with mise:
mise use -g ubi:akitaonrails/ai-jail

# bubblewrap on Linux (pick your distro)
sudo pacman -S bubblewrap # Arch
sudo apt install bubblewrap # Debian / Ubuntu
sudo dnf install bubblewrap # Fedora

Check the binary is there and that bwrap runs without root (it must not ask for sudo):

ai-jail --version
bwrap --ro-bind / / --unshare-all echo "bwrap ok without root"

If bwrap fails asking for privileges, your kernel has unprivileged user namespaces disabled; enable them (sysctl kernel.unprivileged_userns_clone=1 on older Debian/Ubuntu) before continuing.

A1 — The per-project .ai-jail file

On the first launch inside the project, ai-jail creates a .ai-jail (TOML) that is committable to the repo: any colleague who clones inherits the same policy.

cd ~/Projects/mi-app
ai-jail claude # creates .ai-jail and launches Claude Code inside the sandbox

The generated file:

# .ai-jail — sandbox configuration (commit it to the repo)
command = ["claude"]
rw_maps = [] # extra writable directories
ro_maps = [] # extra read-only directories

Before trusting the sandbox, audit it. --dry-run --verbose prints every mount point, every isolation flag and the full bwrap command, without executing anything:

ai-jail --dry-run --verbose claude

Read the output and confirm three things: that $HOME is mounted as a tmpfs (not the real one), that ~/.ssh, ~/.aws and ~/.gnupg do not appear among the mounts, and that the only writable directory is the project one. If you need an extra directory:

ai-jail --rw-map ~/Projects/shared-lib claude # extra, writable
ai-jail --map /opt/datasets claude # extra, read-only

Other agents, same binary:

ai-jail codex
ai-jail opencode
ai-jail bash # bare shell to debug the sandbox
ai-jail -- python script.py # any command

A2 — Permission allowlists with –bootstrap

--bootstrap generates each agent’s permission configuration, with sensible allow/deny/ask, and takes a backup before overwriting:

ai-jail --bootstrap

What it produces, in summary:

Agent	File	Base policy
Claude Code	`~/.claude/settings.json`	allow: `git status/diff/log`, `ls`, `grep`, `cargo`, `npm`, `python`, `docker compose` · ask: `git push`, `rm`, `docker run` · deny: `rm -rf`, `sudo`, `chmod 777`, `git push --force`
Codex	`~/.codex/config.toml`	`approval_policy = "on-request"`
OpenCode	`~/.config/opencode/opencode.json`	permissions for `bash`, `edit`, `write`

The operational key: git push is under ask, not allow, and git push --force under deny. The agent can commit, branch and rebase locally as much as it wants; none of that touches the remote. (If you use the Claude Code /sandbox, also set "allowUnsandboxedCommands": false to close the dangerouslyDisableSandbox escape hatch, which is opt-out out of the box.)

A3 — Lockdown for anything you do not trust

To audit third-party code or run an agent over a project you do not know, --lockdown goes further: project mounted read-only, GPU/Docker/display disabled, --rw-map/--map ignored, $HOME a pure tmpfs with no host dotfiles, network cut with --unshare-net and the environment wiped with --clearenv.

ai-jail --lockdown bash

It is the most restrictive sandbox possible short of a VM. Use it as your mental default for anything that is not your own code on your own machine.

A4 — The safety net: git with no push

It is not a flag, it is a property of the environment that changes the risk arithmetic. If the project is in git with a remote, and the agent does not have push permission, the worst case, that it corrupts every file in the project, is reverted with:

git checkout . # back to the last commit
# and if it touched .git (unlikely): delete the dir and re-clone

The remote was never touched. Sandbox for the filesystem + git for the code + manual push is already a reasonable level for daily use: ai-jail protects your data and the system, git protects the code, and the decision to publish is still yours.

Track B — Cluster (RKE2 with Cilium + Tetragon)

The untrusted agent, or the inference that executes generated code, runs as a pod. The same principle as on the client, different primitives. We assume a generic RKE2 cluster with Cilium as CNI and Tetragon already deployed (the eBPF agent DaemonSet on every node).

B0 — The pod baseline

Before any eBPF, the standard kit. Unprivileged securityContext, read-only root, default seccomp:

apiVersion: v1
kind: Pod
metadata:
 name: ai-agent
 namespace: agentes
 labels:
 app: ai-agent
spec:
 securityContext:
 runAsNonRoot: true
 runAsUser: 10001
 seccompProfile:
 type: RuntimeDefault
 containers:
 - name: agent
 image: registry.interno/ai-agent:pinned
 securityContext:
 allowPrivilegeEscalation: false
 readOnlyRootFilesystem: true
 capabilities:
 drop: ["ALL"]
 volumeMounts:
 - { name: work, mountPath: /work }  # the only writable one
 volumes:
 - name: work
 emptyDir: {}

And the default egress cut, the cluster twin of --unshare-net. A default-deny egress NetworkPolicy in the namespace, opening only DNS and the bare essentials:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
 name: default-deny-egress
 namespace: agentes
spec:
 podSelector: {}
 policyTypes: ["Egress"]
 egress:
 - to:
 - namespaceSelector:
 matchLabels: { kubernetes.io/metadata.name: kube-system }
 ports:
 - { protocol: UDP, port: 53 }
 - { protocol: TCP, port: 53 }

B1 — Kata RuntimeClass: the untrusted pod in its own microVM

For genuinely untrusted code, take it out of the shared kernel. With Kata deployed there is a RuntimeClass:

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
 name: kata
handler: kata

And the pod asks for it with one line, runtimeClassName: kata, running in its own microVM with a dedicated kernel instead of sharing the node’s:

spec:
 runtimeClassName: kata  # ← the pod runs in a microVM, not on the node's kernel
 # ...the rest the same as B0

It is the cluster twin of isolation by construction: a kernel exploit inside the pod does not reach the node.

B2 — Tetragon, observation phase (Post)

Now the layer that separates a platform with runtime visibility from one without. Observe first, never kill from the outset. A TracingPolicyNamespaced, scoped to the namespace and to the agent’s label, that reports (does not kill) three things: process executions, network connections and opens of sensitive paths. action: Post only emits the event.

apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
 name: agente-observa
 namespace: agentes
spec:
 podSelector:
 matchLabels:
 app: ai-agent
 kprobes:
 # --- outbound connections ---
 - call: "tcp_connect"
 syscall: false
 args:
 - index: 0
 type: "sock"
 selectors:
 - matchActions:
 - action: Post
 # --- opens of sensitive files ---
 - call: "security_file_open"
 syscall: false
 args:
 - index: 0
 type: "file"
 selectors:
 - matchArgs:
 - index: 0
 operator: "Prefix"
 values:
 - "/var/run/secrets"
 - "/work/.git/config"
 matchActions:
 - action: Post

(Process executions need no kprobe: Tetragon emits process_exec/process_exit natively.) Deploy it and watch the events live from the node’s Tetragon pod:

kubectl apply -f agente-observa.yaml
# readable events, filtered by namespace:
kubectl exec -n kube-system ds/tetragon -c tetragon -- \
 tetra getevents -o compact --namespace agentes

Leave this running for a typical working day of the agent. Note which destinations it really connects to (your internal registry, your HF mirror, your vLLM endpoint) and which paths it opens. That is your baseline: the list of what is legitimate. Without this step, a Sigkill kills good work and generates an availability incident, precisely what ENS asks you to avoid.

B3 — Tetragon, enforcement phase (Sigkill)

With the baseline in hand, promote to blocking. Two rules. The first: kill any connection whose destination is not on the allowlist — NotDAddr inverts the match: it fires for everything that is not those networks. The second: kill any attempt to open a secrets path.

apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
 name: agente-enforce
 namespace: agentes
spec:
 podSelector:
 matchLabels:
 app: ai-agent
 kprobes:
 # --- egress: kill everything that is NOT on the allowlist ---
 - call: "tcp_connect"
 syscall: false
 args:
 - index: 0
 type: "sock"
 selectors:
 - matchArgs:
 - index: 0
 operator: "NotDAddr"
 values:
 - "127.0.0.1"
 - "10.0.0.0/8" # internal cluster network
 - "172.16.10.20" # internal registry (example)
 matchActions:
 - action: Sigkill
 # --- reading secrets: kill the process ---
 - call: "security_file_open"
 syscall: false
 args:
 - index: 0
 type: "file"
 selectors:
 - matchArgs:
 - index: 0
 operator: "Prefix"
 values:
 - "/var/run/secrets/kubernetes.io/serviceaccount/token"
 - "/work/.ssh"
 matchActions:
 - action: Sigkill

kubectl apply -f agente-enforce.yaml

Now the agent can do whatever it likes inside the pod, but the instant it tries to connect to a disallowed destination or read the service account token, Tetragon kills it in the kernel, before the packet leaves or the read returns bytes. It is the cluster twin of the curl blocklist and the unmounted ~/.ssh, but applied at runtime and over any binary, not just the ones you know about.

Operational warning. Enforcement with Sigkill requires a recent kernel with eBPF support for the action (5.10+ is safe). Deploy agente-enforce in a test namespace first, and keep agente-observa active in parallel: if the block fires, the Post event tells you exactly what triggered it. Adopt first, block later.

The client ↔ cluster equivalence table

The same vector, the two primitives. This is “extrapolating the technology” made explicit:

Threat vector	Client (workstation)	Cluster (RKE2)
`$HOME` / writable root	`$HOME` as ephemeral tmpfs (`bwrap`)	`readOnlyRootFilesystem: true` + `emptyDir`
Arbitrary egress	`curl`/`wget` blocklist · `--unshare-net`	default-deny NetworkPolicy + Tetragon `NotDAddr`→`Sigkill`
Reading secrets	`~/.ssh`/`~/.aws`/`~/.gnupg` not mounted	secrets outside the pod + Tetragon `security_file_open`→`Sigkill`
Kernel escape	Landlock (2nd VFS barrier)	`runtimeClassName: kata` (microVM, own kernel)
No escape hatch	process inside `bwrap`, no way out	no `privileged`, `drop ALL`, `allowPrivilegeEscalation:false`
Damage to the code	git remote with no `push` → `git checkout .`	GitOps + PR review, the agent does not apply to `main`
Visibility	`--dry-run --verbose` (static, pre-run)	Tetragon `tetra getevents` (dynamic, at runtime)

Checklist of gotchas

Do not put in a Sigkill without passing through Post. The observation baseline is not optional: it is what separates “blocking a C2” from “killing your own fine-tuning job”.
The .ai-jail gets committed; secrets do not. The TOML is policy, not credentials. Check you are not putting paths holding sensitive data into rw_maps.
readOnlyRootFilesystem breaks apps that write to /tmp. Mount an emptyDir at /tmp as well as the work one.
A NetworkPolicy with no DNS rule leaves the pod blind. Open port 53 to kube-system or nothing resolves.
Kata is not free. It adds startup latency and not every workload with special devices (GPU passthrough) fits; reserve it for the untrusted, not for everything.
The Claude Code /sandbox does not cover MCP or hooks unless you enable sandbox-runtime. If your agent uses MCP servers, assume they run with full permissions until you do.
NotDAddr with literal IPs ages badly. Document the allowlist and review it when the registry or the inference endpoint changes; consider stable internal CIDRs instead of loose IPs.

References

ai-jail (Fabio Akita), GPL-3.0: https://github.com/akitaonrails/ai-jail
bubblewrap: https://github.com/containers/bubblewrap
Landlock LSM: https://landlock.io
Tetragon — TracingPolicy: https://tetragon.io/docs/concepts/tracing-policy/
Tetragon — enforcement (Sigkill/Override): https://tetragon.io/docs/concepts/enforcement/
Kata Containers — Kubernetes RuntimeClass: https://katacontainers.io
Kubernetes — Pod Security & seccomp: https://kubernetes.io/docs/tutorials/security/seccomp/
Kubernetes — Network Policies: https://kubernetes.io/docs/concepts/services-networking/network-policies/
Cilium: https://cilium.io

Networkpolicy on lo0 — Tech Blog

Hardening and secrets in the sovereign LLM stack: defence in depth

TL;DR

The analogy: the sovereign office

Layer 1 — Secrets: GitOps’s chicken-and-egg problem

Family A — sealed-secrets (asymmetric encryption, controller in the cluster)

Family B — SOPS + age (file encryption, simple)

Family C — External Secrets Operator + Vault (dynamic secrets, native rotation)

How to choose, and rotation

Layer 2 — Network: default-deny, egress and the arithmetic of the surface

The arithmetic of communication pairs

Standard L3/L4, L7 and egress with Cilium

Internal mTLS

Layer 3 — Pod security: restricted, and nothing privileged

Layer 4 — Image supply chain

Layer 5 — AuthN/Z: virtual keys, OIDC and minimal RBAC

Layer 6 — Runtime: the cameras with Tetragon

Layer 7 — Data at rest

Exposure table: service × port × external egress?

Blast radius: a leaked secret, with and without defence

The ENS / NIS2 angle

Applied to the generic 4×H100 cluster

What hardening does NOT solve

See also

Runbook: caging the AI agent — bubblewrap on the client, Tetragon on the cluster

TL;DR

The flow of the two tracks

Track A — Client (the developer’s workstation)

A0 — Install ai-jail and bubblewrap

A1 — The per-project .ai-jail file

A2 — Permission allowlists with –bootstrap

A3 — Lockdown for anything you do not trust

A4 — The safety net: git with no push

Track B — Cluster (RKE2 with Cilium + Tetragon)

B0 — The pod baseline

B1 — Kata RuntimeClass: the untrusted pod in its own microVM

B2 — Tetragon, observation phase (Post)

B3 — Tetragon, enforcement phase (Sigkill)

The client ↔ cluster equivalence table

Checklist of gotchas

See also

References