AI Infrastructure on lo0 — Tech Blog

Storage in the AI era (1/4): the state of the art

Mon, 22 Jun 2026 12:00:00 +0200

Public conversation about AI infrastructure almost always revolves around the GPU: how many FLOPs, how much HBM, how many watts. But anyone who designs or runs an AI factory knows the bottleneck has moved. A Blackwell GPU sitting idle waiting for data costs exactly the same as a Blackwell GPU doing work; what makes the difference is the subsystem feeding it. This first article in the series maps the state of the art in AI storage as of mid-2026: the memory and storage hierarchy, the media technologies (NAND, NVMe, CXL), the parallel file systems that dominate GPU clusters and the new inference stack that has turned the SSD into a component of the memory hierarchy.

The three articles that follow go deeper into the three properties that matter to an architect once that storage reaches production: performance, security and availability.

The underlying problem: the memory wall

A modern GPU is, above all, a machine for moving data. Compute itself is almost never the limit; the limit is the bandwidth feeding the functional units. The hierarchy that runs from GPU registers down to NVMe spans six orders of magnitude in latency and several in bandwidth. Registers are on the order of a million times faster than an NVMe access, and in between sit on-chip SRAM, HBM, host DRAM and, finally, persistent storage.

The practical consequence is well known: in many training workloads GPUs run at 30-50 % of their theoretical FLOPs utilisation because they spend a significant fraction of their time waiting for data. This is the memory wall. Its most useful quantitative formulation is arithmetic intensity, the ratio of operations to bytes moved:

$$I = \frac{\text{FLOPs}}{\text{bytes}}$$

When a workload’s intensity falls below the hardware’s break-even point (the ridge point of the roofline model), performance is limited by bandwidth rather than compute. The industry’s race towards HBM, towards NVLink and, at the layer that concerns us, towards 28 GB/s NVMe and 800 Gb/s networks, is at bottom a war against that wall.

The problem gets worse because compute has grown much faster than memory bandwidth. Each GPU generation multiplies FLOPs more generously than the TB/s of memory that feed them, so the ridge point shifts towards ever higher arithmetic intensities: more workloads fall on the bandwidth-limited side. This is the structural reason storage and memory have moved to the foreground. It is not that storage has improved less (it has improved enormously), but that the GPU demands more of it every year. An architect who designs with FLOPs alone in mind will build systems that spend half their time waiting.

It pays to keep the whole pyramid in view when we talk about “storage” in AI, because design decisions at one level constrain the others:

Level	Technology	Typical bandwidth	Latency	Capacity per device
On-package	HBM3e / HBM4	1.2-3.3 TB/s per stack	ns	36-64 GB
Host memory	DDR5 / CXL	0.3-0.5 TB/s	~100 ns	TB
Hot flash	NVMe Gen5/Gen6	14-28 GB/s	10-100 µs	4-256 TB
Capacity	Nearline QLC / HDD	1-7 GB/s	ms	30-245 TB
Archive	Object / tape	variable	s	EB (aggregate)

The state of the art in 2026 consists, to a large extent, in how the boundaries between these levels blur: HBM spills over into DRAM, DRAM into CXL, and the inference KV-cache into NVMe. Storage has stopped being the basement of the hierarchy and has become part of it.

HBM4: the ceiling rises, but it stays expensive

At the top sits high-bandwidth memory stacked next to the GPU. In April 2025 JEDEC published the HBM4 standard (JESD270-4), which doubles the interface to 2048 bits per stack against HBM3e’s 1024 bits, reaches up to 8 Gb/s per pin and lands around 2 TB/s typical with peaks up to 3.3 TB/s per stack. It supports configurations of 4 to 16 layers with 24 or 32 Gb dies, up to 64 GB per cube. HBM3e, the generation that dominates accelerators in production in mid-2026, sits near 1.2 TB/s and a 36 GB maximum.

The figure that matters to an architect is not peak bandwidth but scarcity. Micron has essentially sold out its entire HBM production through 2026, and capacity expansion is not expected before late 2027-2028. That constraint is what pushes the load downwards through the hierarchy: if you cannot add HBM, you have to learn to spill gracefully into DRAM, CXL and flash. Much of the recent innovation comes from there.

NAND and SSD: capacity explodes, the interface widens

The flash layer is where AI’s pressure on the market shows most. Two vectors at once: capacity and interface.

On capacity, 2025-2026 has been the year of the race to 245 TB. The Solidigm D5-P5336 reached 122 TB (PCIe 4.0) and became the highest-capacity SSD available, on sale for around 12,400 USD; Solidigm has confirmed models of 245 TB or more for late 2026. Micron introduced the 6600 ION with 245 TB in E3.L form factor. By mid-2026 there are at least eight 245 TB SSDs announced by SanDisk, Samsung, Kioxia, Micron, SK hynix and others, all built on high-density QLC NAND (SK hynix is already producing 321-layer QLC) to satisfy hyperscaler demand. Enterprise QLC is consolidating as a replacement for nearline HDD, whose supply lags well behind demand and whose lead times have stretched.

QLC deserves a qualification that sometimes gets lost in the excitement over capacity: write endurance. QLC NAND stores four bits per cell, which multiplies density but reduces the number of erase cycles it tolerates compared with TLC. For AI datasets written once and read many times, the typical case of a training set, this is irrelevant and QLC is the right choice. For workloads with intensive, constant writes, certain checkpoints or a fast-rotating KV-cache, wear has to be watched and TLC often reserved. Knowing the write pattern of each workload is what allows the right medium to be assigned without overpaying or under-wearing.

On interface, PCIe Gen6 has entered datacenter production. The Micron 9650 was the first mass-produced PCIe Gen6 datacenter SSD (February 2026), with 28 GB/s of throughput, roughly double Gen5. Controllers are keeping pace: the Silicon Motion SM8466 advertises 28 GB/s sequential and 7 million random IOPS; FADU’s Gen6 part (“Lhotse”) targets 28.5 GB/s read, 6.9 million IOPS, NVMe 2.2 and native SR-IOV with 128 virtual functions. The adoption path is clear: Gen6 arrives first in AI, HPC and the datacenter, and only afterwards, if at all, in the consumer PC.

Over the network, NVMe-oF splits into two worlds. NVMe/TCP leads greenfield array-to-host deployments for its operational simplicity, with latencies of 100-200 µs. NVMe over RDMA (RoCEv2, iWARP) remains the gold standard for low latency (10-20 µs) where a lossless fabric already exists, such as the ones AI clusters deploy. Ethernet at 25-100 Gb/s is steadily eroding Fibre Channel’s share of storage traffic.

The choice between TCP and RDMA is not an aesthetic preference. NVMe/TCP runs over any existing Ethernet network and greatly simplifies operations, at the cost of higher latency and of host CPU consumption that, under heavy load, competes with compute. NVMe/RoCE demands a lossless fabric, with priority flow control and congestion control properly tuned, plus capable NICs, but it delivers the latency that latency-sensitive AI workloads need and offloads processing to the NIC. In an AI cluster that already deploys InfiniBand or RoCE for GPU-to-GPU traffic, reusing that fabric for storage is the natural decision; in a mixed enterprise infrastructure, NVMe/TCP usually wins on operability.

CXL: from PowerPoint to production

Compute Express Link spent years as the eternal promise. By early 2026 it has finally reached mainstream adoption in production platforms with CXL 3.0/3.1, aligned with PCIe 6.1. The hardware support is there: Intel Xeon 6 (Granite Rapids and later) includes native CXL 3.0 in the CPU, and AMD EPYC Turin is a target platform. CXL 3.0 multi-host pooling switches are already running at some colocation providers.

The use case that has moved from theory to numbers is memory tiering: keeping hot data in GPU HBM and pushing cold data to a shared CXL DRAM pool. Supply chain vendors (Astera Labs, MemVerge, XConn) report notable gains from offloading the inference KV-cache to CXL memory compared with SSD or RDMA-based solutions. The specific figures deserve caution, since they come largely from the manufacturers themselves, but the direction is real: CXL has become a usable disaggregated memory layer, not a laboratory experiment. Tiering support in the Linux kernel (6.1 onwards) is maturing, although getting value from it still requires NUMA-aware applications.

Parallel file systems: the GPU battlefield

If the media layer is where silicon vendors compete, the parallel file system layer is where platform vendors compete, and it is the one an AI architect touches daily. The market has reshuffled in a few years.

WEKA has probably been the most talked-about case. Its system (WekaFS, now repackaged as NeuralMesh) bypasses the Linux kernel by running as an RTOS in user space, uses RDMA instead of TCP/IP and distributes metadata through consistent hashing. Published figures cite latencies of 100-200 µs against more than 1000 µs for traditional stacks, and metadata performance that doubles Lustre’s. In 2026 it launched its NeuralMesh AI Data Platform on the NVIDIA reference design, with an “Augmented Memory Grid” aimed at context memory for agentic AI.

VAST Data has bet on a different architecture, DASE (Disaggregated Shared-Everything), a single all-flash tier on which it has built what it calls an “AI OS”: a stack with a database, a data engine, InsightEngine for real-time AI pipelines, vector search and RAG, and integrated governance. Azure has adopted its system, a sign of the traction it has gained.

Lustre (DDN) remains the most widespread file system in classic supercomputing, but it is being eroded in the GPU arena by VAST and WEKA. DAOS shines at the top end of traditional HPC but barely appears in commercial AI clusters. IBM Storage Scale (the former Spectrum Scale/GPFS) has repositioned itself with Content Awareness for RAG and support for NVIDIA’s data platform, alongside a refresh of the FlashSystem range.

The common denominator of this whole generation is alignment with NVIDIA. Platforms are certified against DGX SuperPOD and integrate with GPUDirect Storage, which allows direct DMA between storage and GPU memory, skipping the CPU and the bounce buffer. In GB200 NVL72 architectures, a liquid-cooled rack that presents 72 GPUs as a single giant GPU through NVLink, with 13.4 TB of unified GPU memory, GPUDirect Storage certification stops being an ornament and becomes a requirement.

To place the main players side by side, it helps to summarise their architectural bets, which are genuinely different and not mere variations on a single design:

Platform	Architecture	Differentiator	Fit in AI
WEKA NeuralMesh	User-space RTOS, RDMA	Very low latency, metadata distributed by hashing	Training and agentic inference, context memory
VAST DataStore	Disaggregated all-flash DASE	Single exabyte-scale multiprotocol namespace, AI OS	Lakehouse + RAG + inference
IBM Storage Scale	Parallel (ex-GPFS)	Content Awareness for RAG, HPC maturity	Enterprise and HPC with unstructured data
DDN EXAScaler/Lustre	Hardened Lustre	Raw throughput per rack, massive deployments	Supercomputing and AI factories
Pure Storage	All-flash with Purity	Evergreen operations, simplicity	Enterprises that value operations over peak

The choice between them is rarely decided by peak GB/s, but by workload pattern (sequential training against random inference), by the operating model and by the degree of integration required with the NVIDIA stack. A sensible architect first assesses their workload mix and only then compares numbers.

The rack as the unit of compute: GB200, Rubin and BlueField-4

The change in granularity is worth pausing on. For decades the design unit was the server; today, in AI, it is the rack. The GB200 NVL72 integrates 72 Blackwell GPUs and 36 Grace CPUs into a single 130 TB/s NVLink domain, presented to software as one logical GPU with 13.4 TB of unified memory and capable of up to 30 times the real-time inference of a trillion-parameter LLM against the previous generation. At that scale storage does not compete with internal bandwidth (it could not), but it must feed model loading and the inference KV-cache (and, where applicable, training checkpointing) without strangling the 72 GPUs that behave as one.

The roadmap sharpens the trend. The Rubin generation and, in particular, the Rubin CPX, a GPU specialised in long-context inference with more than a million tokens, push still more data towards the memory and storage layers. And the piece that stitches it all together is the DPU: NVIDIA’s BlueField-4 enables an “Inference Context Memory Storage” platform that turns the KV-cache into a pod-level shared flash resource, connected over Spectrum-X Ethernet. The DPU offloads storage and security data movement from the CPU, and is becoming the true storage controller of the AI node. For the architect, this means storage design no longer ends at the array: it includes the NIC, the DPU and the fabric.

The inference stack: when the SSD joins the memory hierarchy

The most important conceptual change of the past two years, and the one that best summarises the state of the art, is this: flash storage has become part of the inference memory hierarchy. The cause is context windows. As models handle hundreds of thousands, or millions, of tokens, the KV-cache generated by the prefill phase overflows HBM. That overflowing KV-cache drops first into host DRAM and then to the NVMe SSD.

NVIDIA has standardised this pattern. Its Dynamo orchestration layer operates across the whole hierarchy, HBM → CPU DRAM → NVMe → network storage, with a KV-cache offload engine and a transfer library (NIXL) for sharing cache between nodes. In January 2026 it standardised KV-cache offload to NVMe SSD, and it has introduced an “Inference Context Memory Storage” platform based on BlueField-4 DPUs that turns the KV-cache into a high-bandwidth shared resource at pod level. Solidigm and other SSD makers have aligned their message around the same idea: long-context inference is, in part, a flash storage problem.

The design consequences are concrete. A 2026 inference server carries between 8 and 32 SSDs, with the figure expected to reach 32 within a couple of years, and a random access pattern, against the sequential pattern and the dozen or so drives of a training server. The ratio of inference to training servers can reach 50 to 1. The centre of gravity of storage investment is shifting towards inference.

Anatomy of storage in an inference factory

It is worth pausing here, because an inference AI factory, a system designed to serve models in production rather than train them, has a storage profile of its own, different from the training cluster that dominates the literature. While training is a batch job that tolerates stoppages and is measured in weeks, inference is a continuous service, latency-sensitive and bound by an SLA: every millisecond and every failure shows up at the client. Its storage is organised into four planes that are best designed separately.

The first plane is the model registry. The weights of the models being served, which may number in the dozens, with versions, quantised variants and LoRA adapters, live in object storage or a shared file system, and are loaded into GPU HBM when a model is activated. The critical requirement here is not sustained throughput but load time: when a new replica has to start or a GPU has to switch models, reading tens or hundreds of GB of weights determines the cold start. A slow registry translates into sluggish autoscaling and GPUs that take their time entering service, exactly when demand rises.

The second plane is the KV-cache hierarchy, the heart of modern inference performance. As we saw, the KV-cache overflows HBM and drops to DRAM, CXL and NVMe. In an inference factory this is not a marginal detail: it is a first-class storage layer, with its own sizing, its random access pattern and critical latency, and a lifetime measured in seconds. NVIDIA’s standardisation of KV-cache offload to NVMe SSD and the ICMS platform on BlueField-4 exist precisely for this: to allow long contexts and cache reuse across requests (and across nodes) without paying the price of recomputation or of adding HBM, which is scarce and expensive.

The third plane is the knowledge store for RAG: the vector databases (Milvus, Weaviate, Pinecone, or S3 Vectors) and the source documents the system retrieves to enrich its answers. The pattern here is read-intensive, with similarity searches over indexes that can run to TB and are best kept on low-latency flash. The availability and freshness of this store determine answer quality as much as the model itself, and updating it, reindexing and taking in new documents, is a constant write flow the design has to account for.

The fourth plane is observability and logging: traces, request and response logs, metrics and, where regulation demands it, the auditable archive of interactions. It is a high-volume append-only write flow that usually ends up in object storage, and one that a GPU-centric design tends to forget about until it saturates something.

The conclusion for anyone building an inference factory is that its storage looks nothing like a rescaled training cluster. It is sized by model load time, by KV-cache capacity and latency, by vector search performance and by telemetry volume, with a cross-cutting imperative of continuous service under SLA. Training, where it exists, is the occasional guest; inference is the permanent resident.

The other end: object, lakehouse and vectors

Not everything is hot flash. The training database and the system of record live in object storage. The novelty of 2025-2026 is convergence with open table formats: Apache Iceberg, Delta Lake and Hudi bring ACID transactions, time travel and schema evolution on top of cheap object storage. AWS has launched S3 Tables (managed Iceberg, with faster queries) and S3 Vectors (native vector storage with savings of up to 90 % for AI workloads). Vector databases, Pinecone, Milvus, Weaviate, integrate with data lakes to feed RAG, and columnar formats such as Parquet, together with Lance for multimodal data, are consolidating as the substrate of the lakehouse.

Computation near the data (computational storage) remains active R&D rather than mass deployment. The corresponding NVMe specifications have been published since late 2023, with eBPF as the execution environment for downloadable functions, but real adoption is still marginal. The idea, running filtering, decompression or search directly on the device so the data never travels to the CPU, is seductive for AI workloads that read enormous datasets at low arithmetic intensity, but the programming model and the lack of tooling standardisation hold deployment back. It is a technology to watch, not yet one to put into production.

The data lifecycle: tiering as an architectural decision

Taking in the whole picture, it is clear that no data lives in a single place for its entire lifecycle. A dataset arrives raw in the object data lake, is transformed and catalogued in an open table format (Iceberg, Delta), is promoted to the hot flash layer of the parallel file system to feed training, generates checkpoints that live for days on NVMe, and finally, together with the resulting models, is archived to cold object storage or tape. Automatic tiering between these layers has gone from a cost optimisation to a first-order architectural decision, especially amid NAND and HBM scarcity.

The practical rule that emerges is to place every byte in the cheapest tier that satisfies its latency and bandwidth requirement at that moment, and to automate the transitions. Hot flash (NVMe Gen5/Gen6) absorbs the training working set and the inference KV-cache; capacity QLC replaces nearline HDD for datasets that are read sequentially; object storage is the durable, versioned system of record; and tape is resurging as an active archive at exabyte scale to fight the cost of flash. Designing those boundaries, and measuring how much data crosses each one, is now part of the AI storage architect’s job, not the administrator’s.

The market context: a supply supercrisis

None of this happens in an economic vacuum. Storage demand from AI servers is 8 to 10 times that of a traditional server, and it is growing above 20 % a year, while global NAND supply grows only 15-17 %. The result, in the words of several analysts, is a memory “supercrisis”: high double-digit contract price rises for DRAM and NAND in 2026, stretched lead times and collateral effects on smartphones and PCs. For an architect, this turns capacity efficiency (QLC, compression, erasure coding, tiering) into a first-order cost lever rather than an implementation detail.

What this changes for the architect

If the state of the art had to be distilled into a handful of practical implications, these would be them. First, storage is designed by workload pattern, not by capacity: a training server and an inference server have opposite requirements, sequential against random, few drives against many, and mixing them in a single template is a common mistake. Second, the limit almost never sits in the medium any more but in the full chain of medium, NVMe, DPU, NIC, fabric and file system; optimising one link while leaving the rest untouched does not move the needle. Third, supply scarcity turns capacity efficiency (QLC, compression, erasure coding, tiering) into a cost lever comparable to buying the drives themselves. And fourth, the boundary between memory and storage has dissolved: thinking in terms of “RAM on one side, disk on the other” no longer describes a modern AI system, where HBM, DRAM, CXL and NVMe form a software-managed continuum.

Takeaways

The state of the art in AI storage in 2026 comes down to one idea: the hierarchy has become continuous. HBM spills into DRAM, DRAM is disaggregated over CXL, the inference KV-cache drops all the way to NVMe, and the parallel file system orchestrates the whole so the GPU never waits. Media technologies (HBM4, 245 TB QLC, NVMe Gen6) raise the ceilings, but supply scarcity turns efficiency into a design constraint. And the software battlefield, WEKA, VAST, IBM, DDN, is fought over a single metric: keeping the GPUs fed.

With this map on the table, the three articles that follow go into the detail of the properties that decide whether a storage architecture works in production.

Sources

FADU, From Training to Inference: FADU at CFMS 2026 — https://blogs.fadu.io/ai-inference-ssd-data-centers/
Avnet, Riding the AI Supercycle: 2026 Memory & Storage Market — https://www.avnet.com/integrated/resources/article/2026-memory-shortage-ai-supercycle/
JEDEC, JESD270-4 HBM4 Standard — https://www.jedec.org/news/pressreleases/jedec%C2%AE-and-industry-leaders-collaborate-release-jesd270-4-hbm4-standard-advancing
Tom’s Hardware, JEDEC finalizes HBM4 memory standard — https://www.tomshardware.com/pc-components/ram/jedec-finalizes-hbm4-memory-standard-with-major-bandwidth-and-efficiency-upgrades
StorageNewsletter, Micron 9650 PCIe Gen6 data center SSDs in mass production — https://www.storagenewsletter.com/2026/02/19/micron-9650-pcie-gen6-data-center-ssds-in-mass-production/
Tom’s Hardware, Solidigm reveals 122TB SSD — https://www.tomshardware.com/pc-components/ssds/solidigm-reveals-122tb-ssd-the-worlds-highest-capacity-drive-for-ai-workloads-d5-p5336-offers-unlimited-write-durability
TechRadar, Solidigm confirms 245TB SSDs before end of 2026 — https://www.techradar.com/pro/solidigm-confirms-245-tb-ssds-set-to-launch-before-end-of-2026
KAD, CXL in 2026: How Memory Pooling is Reshaping Data Centers — https://www.kad8.com/hardware/cxl-in-2026-how-memory-pooling-is-reshaping-data-centers/
Blocks & Files, Parallel filesystem definitions — https://blocksandfiles.com/2025/11/26/parallel-filesystem-definitions-and-powerscale/
NAND Research, WEKA NeuralMesh AIDP & STX integration — https://nand-research.com/weka-neuralmesh-aidp-stx-integration-gtc-2026/
VAST Data, InsightEngine — https://www.vastdata.com/platform/insightengine
NVIDIA, GB200 NVL72 — https://www.nvidia.com/en-us/data-center/gb200-nvl72/
Blocks & Files, Nvidia standardizes GPU cluster KV cache offload to NVMe SSDs — https://blocksandfiles.com/2026/01/06/nvidia-standardizes-gpu-cluster-kv-cache-offload-to-nvme-ssds/
AWS, Building AI-Ready Data Lakes: S3 Tables, Iceberg, S3 Vectors — https://builder.aws.com/content/34vssNUviyG5WPv3IRiGZFKZYkn/building-ai-ready-data-lakes-amazon-s3-tables-apache-iceberg-and-s3-vectors
TrendForce, AI Server Demand to Drive Memory Contract Price Increases 2Q26 — https://www.trendforce.com/presscenter/news/20260331-12995.html

Storage in the AI era (2/4): performance

Mon, 22 Jun 2026 11:00:00 +0200

In the first article we drew the map: the memory-storage hierarchy, the media technologies and the software stack that dominates GPU clusters. This second article goes into the property that justifies all that complexity: performance. The question an architect has to answer is not “how many GB/s does my array deliver?”, but “can I get my GPUs working at 95 % instead of 40 %?”. That difference is, literally, half the compute bill.

The metric that really matters: accelerator utilisation

It is tempting to measure AI storage with the classic metrics, sequential throughput, random IOPS, latency, and we do need them. But none of them captures what the business pays for. The integrating metric is accelerator utilisation: what fraction of the time the GPU spends computing rather than waiting for data.

Most real ML pipelines run at between 30 % and 60 % GPU utilisation. Industry surveys are damning: only around 7 % of AI/ML teams manage to exceed 85 % at peak, and poor forecasting or weak autoscaling can leave GPUs idle 70-85 % of the time. A job running at 30 % utilisation is paying for 70 % of its compute capacity and getting nothing for it.

We can formalise this. If $ t_{\text{cmp}} $ is the compute time per batch and $ t_{\text{io}} $ the time the GPU spends blocked waiting for I/O that fails to overlap with compute, effective utilisation is:

$$U = \frac{t_{\text{cmp}}}{t_{\text{cmp}} + t_{\text{io}}}$$

The goal of all AI storage engineering is to drive $ t_{\text{io}} $ towards zero, whether by increasing bandwidth, reducing latency or overlapping I/O with compute through prefetching and pipelining. And there is an aggravating factor: the faster the GPU, the harder it is to feed. Moving from V100 to H100 cut the per-batch compute time of the 3D U-Net model by 76 %, turning a bandwidth-sensitive workload into a latency-sensitive one. Every accelerator generation raises the bar for storage.

This change of regime, from bandwidth-limited to latency-limited, has a design consequence worth internalising. When a workload is bandwidth-limited, the fix is to add transfer capacity: more NVMe, more lanes, more nodes. When it becomes latency-limited, that recipe stops working; what matters then is response time per operation, which is attacked with cache, with intelligent prefetching and with data paths that remove hops (GPUDirect, for instance). Diagnosing correctly which of the two regimes each workload sits in is the first task of any optimisation: throwing bandwidth at a latency-limited workload is spending money without moving utilisation.

MLPerf Storage: the benchmark that does measure the right thing

For years there was no neutral way to compare AI storage platforms. MLPerf Storage, from MLCommons, has provided one. Its design insight is not to measure GB/s in the abstract but to simulate the accelerators’ think time so as to generate a realistic I/O pattern, and to require those simulated accelerators to sustain a minimum utilisation level. In other words, it measures exactly what matters: how many accelerators a platform can keep fed without their utilisation dropping below the threshold.

v1.0 (September 2024) used the 3D U-Net, ResNet-50 and CosmoFlow models, simulating A100 and H100, with utilisation thresholds of 90 % for the first two and 70 % for CosmoFlow. v2.0 (August 2025) marked a leap: more than 200 results from 26 organisations, including Alluxio, DDN, Hammerspace, HPE, IBM, KIOXIA, Micron, Oracle, Samsung and WDC, systems able to sustain roughly twice as many accelerators as in v1.0, and, above all, a new checkpointing workload (which we will come back to).

Some v2.0 results help calibrate the state of the art:

System	Throughput	Accelerators	GPU utilisation
Volumez	1.079 TB/s	—	92.21 %
Hammerspace (5 nodes)	420.8 GB/s	140	>94.7 %
Hammerspace (3 nodes)	253.1 GB/s	84	>94.7 %
Hammerspace (1 node)	85.6 GB/s	28	>94.7 %
Alluxio	24.14 GiB/s	128	99.57 %

The Hammerspace figure illustrates a property an architect should demand: linear scaling. From 1 to 5 nodes throughput grows from 85.6 to 420.8 GB/s while keeping utilisation above 94.7 % with a minute coefficient of variation (0.08-0.14 %). Linear scaling is what separates an AI platform from a fast NAS.

The numbers from commercial platforms

Outside the benchmark, platforms certified for DGX SuperPOD publish aggregate figures worth keeping as an order-of-magnitude reference:

WEKA WEKApod Nitro: 720 GB/s read and 186 GB/s write per configuration, 18 million IOPS, with ConnectX-8 NICs at 800 Gb/s. Per node, 70 GB/s read and 40 GB/s write; the minimum 8-node configuration delivers 560 GB/s read. A single entry-level system covers the I/O demand of a 1,152-GPU GB200 scalable unit.
DDN AI400X2: more than 90 GB/s and 3 million IOPS per appliance; the Turbo variant saturates close to 100 GB/s (800 Gbps) per DGX B200. DDN’s new platform claims more than 1 TB/s read per appliance with per-rack scaling, and has fed NVIDIA’s Eos supercomputer at 4 TB/s.
VAST Data: multiple TB/s from a single mount point via NFS multipathing, RDMA over NFS and GPUDirect, managing exabytes in a single multiprotocol namespace.
Pure Storage: certified for DGX SuperPOD with GB200 and GB300.

These figures only mean something in relation to the cluster they feed. A GB200 NVL72 moves 130 TB/s of internal GPU-to-GPU communication; storage does not compete with that, but it does have to sustain data loading and checkpointing without strangling 72 GPUs that behave as one.

The I/O pattern of training and GPUDirect Storage

Understanding the I/O pattern is the key to neither over- nor under-sizing. Training combines several patterns: large sequential reads of the dataset, small random reads during shuffling between epochs, and massive periodic writes during checkpointing. Each stresses storage differently.

The traditional data path goes through the CPU: from storage to a bounce buffer in host memory and from there to GPU memory. Every byte of every checkpoint and every weight load pays the latency and CPU overhead of that double hop. GPUDirect Storage (GDS) cuts it out: it enables direct DMA between GPU memory and NVMe through the cuFile user-space library and the nvidia-fs kernel module, which intercepts POSIX calls and redirects them through the DMA engine. The result is zero-copy, with direct transfers exceeding 40 GB/s and, above all, without stealing cycles from the CPU.

There is a nuance that naive deployments miss: GDS is optimised for large, aligned, sequential transfers. For workloads dominated by small files or heavily random access, the traditional path with CPU caching can be more efficient. GDS is not a magic “more performance” switch; it is a tool for a specific pattern. Designing the data pipeline, meaning dataset format, shard size and number of DataLoader workers, matters as much as picking the array.

The data pipeline deserves a pause, because it is where performance leaks most quietly. Training does not read the dataset just any way: it walks it in batches, shuffles it between epochs to avoid ordering bias, and often applies transformations (image decoding, tokenisation, augmentation) on the CPU before handing the tensor to the GPU. Each of those steps can become the bottleneck. Too few DataLoader workers leaves the GPU waiting; a dataset format that forces millions of small files open sinks metadata performance; a badly designed shuffle turns efficient sequential reads into costly random accesses. Common practice in serious workloads is to pack the dataset into large shards (formats such as WebDataset, TFRecord or Parquet) precisely so that the read pattern is large and sequential, the terrain where GDS and parallel file systems perform best. The mental rule: the fastest storage in the world will not save a badly designed data pipeline.

The real cost of checkpointing (when there is training)

Checkpointing is the most demanding write workload in the AI world, but it needs to be placed properly: it is a training problem. A pure inference factory barely suffers from it, since its heavy writes are different ones, KV-cache, vector indexes, telemetry. We cover it here because many installations combine some fine-tuning or occasional training with serving, and because it illustrates better than any other case how asynchrony beats brute force. If your platform is for inference, you can read this section as context and jump to the next one, which is the one that really concerns you.

That said, most people’s intuition about checkpointing is wrong on two counts: size and frequency.

Size. A checkpoint is not just the weights. The rule of thumb is around 16 bytes per parameter, because the bulk is optimiser state. The figures from the MLPerf Storage v2.0 checkpointing workload, based on Meta’s Llama 3 Herd, make it plain:

Model	Processes	Checkpoint size	Of which, optimiser state
8B	8	105 GB	90 GB
70B	64	912 GB	—
405B	512	5.29 TB	—
1T	1024	15 TB	13.2 TB

Frequency. At scale, hardware failures are constant (we develop this in the article on availability). Meta’s model for a 16,000-accelerator cluster predicts a failure every few hours. To lose less than 5 % of progress between failures, you need on the order of 20 checkpoints in that interval. That translates into aggressive cadences:

16,000 accelerators: around 155 checkpoints a day, one every 9.3 minutes.
100,000 accelerators: around 967 checkpoints a day, one every 1.5 minutes.

And here is where the bandwidth problem appears. A synchronous checkpoint halts training while it writes. If $ S_{\text{ckpt}} $ is the checkpoint size and $ t_{\text{win}} $ the acceptable time window for writing it while keeping overhead below 5 %, the required bandwidth is:

$$B_{\text{req}} = \frac{S_{\text{ckpt}}}{t_{\text{win}}}$$

For a 1T model on 100,000 accelerators, the window falls to around 4.4 seconds. Writing 15 TB in that time demands on the order of 3.6 TB/s across the cluster, or some 200 GB/s sustained over the whole job. On checkpoints alone, that scenario generates more than 14 PB of writes a day. This is not a marginal workload: it is one of the most demanding write workloads in existence.

It is worth seeing the calculation laid out, because it reveals where the lever is. Take the 405B model, whose checkpoint weighs 5.29 TB. If policy requires writing it while keeping overhead below 5 % at a cadence where the acceptable window is, say, 30 seconds, the synchronous bandwidth needed is around 176 GB/s sustained, within reach of a high-end array, but only one dedicated to that task. If instead of 30 seconds the window contracts to 5 (a larger cluster, more frequent failures), the requirement jumps above 1 TB/s. Sensitivity to the window is brutal: the difference between a viable design and an impossible one lies in how time is managed, not in how many drives are bought. That is where asynchrony changes the rules.

The solution is not just more bandwidth; it is asynchrony. Asynchronous checkpointing decouples the snapshot (a fast GPU→CPU copy) from persistence (a background write, overlapped with compute). PyTorch’s Distributed Async Checkpointing cuts effective checkpoint time by 10 to 20 times; in one IBM example a 7B model brought its down time down from 148.8 to 6.3 seconds, a 23.6-fold improvement. Techniques such as CheckFreq pipeline the snapshot and persistence with training, allowing a checkpoint every 14-19 iterations. This is why vendors like VAST argue the relevant metric is not GB/s but checkpoint overlap: what fraction of the checkpoint overlaps with compute. Around 10 % overlap is usually enough.

The design lesson is clear: sizing training storage for the peak write rate of synchronous checkpointing is expensive and often unnecessary if the software stack gets asynchrony right. But trusting asynchrony without measuring overlap is playing roulette with weeks of compute.

Performance in an inference factory

Here is the heart of the matter for anyone running an inference factory. In an inference factory storage is not measured in epochs or GPU-hours but in the experience the client perceives, and that experience condenses into two metrics: TTFT (time to first token, the time until the first token of the answer) and inter-token latency (the rate at which the following tokens arrive). At fleet scale, the metric that pays the bills is throughput in tokens per second per GPU, because it determines how many requests each accelerator serves. Storage touches all three, in ways a training-centric design does not anticipate.

Prefill, decode and their disaggregation

Inference has two phases with opposite profiles. Prefill processes the entire input prompt at once, generates the key-value pairs of the KV-cache and is a compute-intensive phase; it dominates TTFT. Decode generates tokens one by one, reusing that cache, is memory-bandwidth-intensive and dominates inter-token latency. Because their bottlenecks differ, the 2025-2026 trend is to disaggregate them: run prefill and decode on separate GPU pools and transfer the KV-cache between them. That transfer, which may travel over NVLink, over the network or through a shared storage layer, makes KV-cache movement a performance-critical operation. A factory that disaggregates prefill and decode needs a very high bandwidth, low latency KV-cache path between both pools; storage (or the DPU that governs it) stops being a passive actor and enters the critical path of every request.

The KV-cache as a performance layer

The central problem is the size of the KV-cache with long contexts: it overflows HBM, which is scarce and expensive. The options are to recompute it on every request, ruinously expensive in compute and lethal for TTFT, or to offload it to a slower layer and reuse it. Two approaches compete here. Offload to CXL memory offers gains of more than 5 times against SSD or RDMA-based caching, according to CXL supply chain vendors, and allows batches up to 30 % larger while holding latency targets, which raises throughput and GPU utilisation. Offload to NVMe, standardised by NVIDIA in 2026 with its ICMS platform on BlueField-4, is slower but much cheaper and more capacious, ideal for pod-scale KV-cache.

The big performance lever here is KV-cache reuse (prefix caching). Many requests share prefixes: the same system prompt, the same context document, the same prior conversation. If the KV-cache of those prefixes is kept in a fast storage layer and reused instead of recomputed, prefill shortens drastically and TTFT collapses. This turns a low-latency storage layer into a direct multiplier of the factory’s throughput: the more useful cache fits and the faster it is served, the less compute is wasted recalculating what has already been calculated. Sizing this layer well, in capacity, bandwidth, latency and eviction policy, is one of the most profitable performance decisions in the whole factory.

Model load time and cold start

There is a storage cost specific to inference that training ignores: starting a replica. When autoscaling decides to bring up a new instance of a model, because demand is rising, because a replica has failed or because a version is being rotated, the weights have to be read from the registry and loaded into HBM. For a model of tens or hundreds of GB, that cold start can run from seconds to minutes depending on registry bandwidth and the load path. And during that time the new GPU serves no requests while the system is, perhaps, saturated. A slow model registry translates directly into missed SLAs at peak. The mitigations, keeping weights on hot local flash, using fast-loading formats, streaming weights as they are needed, or GPUDirect Storage to load straight into HBM, are storage decisions that determine the factory’s real elasticity.

RAG performance and vector search

If the factory serves RAG, there is one more link in the critical path of every request: retrieval. Before the model generates anything, the system searches a vector database for the relevant fragments, and that similarity search, over indexes that can run to TB, adds to TTFT. Vector search performance depends on keeping the indexes on low-latency flash and on an efficient random read pattern; an index that does not fit in memory and is served from slow storage adds latency to every request. Beyond that, precomputed KV-caches of the retrieved documents speed up prefill, because the model reuses keys and values already calculated and only encodes the user’s query. An inference factory with RAG therefore has two latency-sensitive read workloads coexisting, the vector index and the KV-cache, competing for the same fast flash and needing to be sized together.

For an inference architect, all this introduces a storage plane with a profile of its own: latency-sensitive random reads (indexes, reusable KV-cache prefixes), high-turnover ephemeral writes (new KV-cache), large, sporadic but urgent reads (model loading) and an append-only telemetry flow. It is not sized in the least like the storage of a training cluster.

How to measure your own: beyond the spec sheet

Vendor and MLPerf figures are useful as a reference, but real performance depends on the specific workload, and an architect must measure rather than assume. The methodology that works starts from three principles.

The first is to measure accelerator utilisation, not storage in the abstract. An array can deliver 500 GB/s in a synthetic test and still leave the GPUs at 50 % because the real pattern is small random files that the sequential test does not reproduce. The right question is always: what fraction of the time is my GPU computing, with this workload, this dataset format and this pipeline?

The second is to reproduce the real I/O pattern, not a convenient one. That means testing with the same dataset format, the same shard size, the same number of workers and the same checkpointing cadence that will be used in production. MLPerf Storage’s insight was precisely to simulate accelerator think time in order to generate a realistic pattern; replicating that rigour at your own scale is what separates a useful measurement from a marketing number.

The third is to look for scaling, not peak. A platform that delivers a very high peak in a small configuration but flattens out as nodes are added is no use for growth. Tests should be run at several scales to verify that throughput grows roughly linearly and that accelerator utilisation holds, as the Hammerspace example in MLPerf v2.0 showed. A low coefficient of variation across configurations matters as much as the peak.

The cost of getting it wrong

It is worth putting a number on the problem. The price of an H100 in the cloud in mid-2026 moves across a wide range by provider, with a market median around 2.3-3.1 USD per GPU-hour. Multiply that by thousands of GPUs and by the weeks a training run lasts, and 40 % utilisation instead of 90 % translates into millions of euros of wasted compute. That is why more than 50 % of organisations report data or storage bottlenecks limiting their AI in 2026, and why storage bandwidth has emerged as a hard ceiling on scaling, comparable to power and cooling: when there is data starvation, adding compute gives diminishing returns.

The operational recommendation that keeps coming up is to size the I/O pipeline for 300 Gbps or more in serious workloads, and to measure, not assume, accelerator utilisation as a health indicator. Optimised storage can deliver up to 5 times the throughput of standard S3 access over HTTP: more than 100 GB/s aggregate against some 20 GB/s.

The physical layer: networks and PCIe Gen6

All of the above rests on a physical layer that has also advanced. NVIDIA’s ConnectX-8 NIC combines PCIe Gen6 with 800 GbE or 800 Gb InfiniBand XDR, designed for Blackwell systems. InfiniBand XDR offers 800 Gbps per direction (double NDR, which remains the usual storage fabric), and the Quantum-X800 switch adds in-network compute that accelerates collective operations. The transition to XDR is accelerating with the Blackwell deployments of 2025-2026. The mental rule is simple: storage must match the network, and the network must match the GPU. One slow link defines the performance of the whole chain.

It is worth translating this into concrete numbers. An 800 Gb/s NIC delivers, at best, around 100 GB/s to a node. If the array can serve more than the NIC absorbs, the network is the bottleneck; if the array serves less, storage is. Balance is found by sizing each layer so none strangles the next, and by verifying it with end-to-end measurements rather than the isolated specifications of each component. It is common to find installations with an excellent array and an undersized fabric, or the reverse, performing far below the sum of their parts. The physical layer also imposes its geography: network topology (how many hops there are between the compute node and the array, whether they share a switch or cross the cluster spine) affects latency as much as the array itself, and an AI storage design that ignores network topology is incomplete.

Three questions before sizing

All this complexity can be organised into three questions an architect should answer before buying anything. What is the dominant I/O pattern of my workload, large sequential, small random, bursty writes, and therefore which metric limits me? Is my workload bandwidth-limited or latency-limited, and consequently where do I invest? And what is my checkpointing budget, in size, cadence and, above all, what fraction can I overlap with compute? Whoever answers these three with measured data, rather than intuitions, sizes well; whoever skips them ends up with idle GPUs or with an oversized, very expensive array that is never put to use. AI storage performance is, at bottom, an exercise in fit: putting the right capacity where the workload needs it, no more and no less.

Takeaways

AI storage performance is not measured in GB/s but in accelerator utilisation, and that utilisation depends on three things: enough bandwidth, low latency and, above all, overlapping I/O with compute. MLPerf Storage v2.0 has finally given a common language for comparing platforms through that lens. Checkpointing is the most demanding write workload and is mastered with asynchrony, not hardware alone. Long-context inference adds a new storage layer around the KV-cache. And the cost of getting it wrong is measured in idle GPUs, which is the most expensive line in the budget.

Performance, however, is worth nothing if the data is neither protected nor available. The third article takes on the security of this new storage layer.

Sources

MLCommons, MLPerf Storage v2.0 Results — https://mlcommons.org/2025/08/mlperf-storage-v2-0-results/
MLCommons, Announcing the MLPerf Storage v2.0 Checkpointing Workload — https://mlcommons.org/2025/08/storage-2-checkpointing/
MLCommons, MLPerf Storage v1.0 Benchmark Results — https://mlcommons.org/2024/09/mlperf-storage-v1-0-benchmark-results/
Hammerspace, MLPerf Storage v2.0 Benchmark Results — https://hammerspace.com/hammerspace-mlperf-storage-v2-0-benchmark-results/
Alluxio, Strong Performance in MLPerf Storage v2.0 — https://www.alluxio.io/blog/alluxio-demonstrates-strong-performance-in-mlperf-storage-v2-0-benchmarks
NVIDIA, GPUDirect Storage Overview Guide — https://docs.nvidia.com/gpudirect-storage/overview-guide/index.html
Spheron, GPU Direct Storage Guide 2026 — https://www.spheron.network/blog/gpu-direct-storage-nvme-ai-training-inference-guide/
PyTorch, Reducing Checkpointing Times with Async Checkpointing — https://pytorch.org/blog/reducing-checkpointing-times/
VAST Data, Optimizing Checkpoint Bandwidth for LLM Training — https://www.vastdata.com/blog/optimizing-checkpoint-bandwidth-for-llm-training
Astera Labs, How CXL Transforms RAG and KV Cache Performance — https://www.asteralabs.com/breaking-through-the-memory-wall-how-cxl-transforms-rag-and-kv-cache-performance/
MinIO, AI Storage Architecture: Overcoming the Bottleneck in 2026 — https://www.min.io/blog/ai-storage-architecture-bottleneck-2026
WEKA, WEKApod Nitro reference architecture — https://www.weka.io/resources/reference-architecture/nvidia-gb200-nvl72-systems-reference-architecture-for-cloud-partners/
Blocks & Files, Storage vendors rally behind Nvidia at GTC 2025 — https://blocksandfiles.com/2025/03/18/nvidia-storage-announcements/
ServeTheHome, NVIDIA ConnectX-8 SuperNIC PCIe Gen6 800G — https://www.servethehome.com/nvidia-connectx-8-supernic-pcie-gen6-800g-nic-detailed/
SpendArk, GPU Cloud Pricing 2026 — https://spendark.com/blog/machine-learning-cloud-cost/

Storage in the AI era (3/4): security

Mon, 22 Jun 2026 10:00:00 +0200

Data is the most valuable and most durable asset of an AI system. Models get retrained; weights get replaced; but training datasets, the customer data feeding RAG and the weights of a frontier model represent years of investment and, often, regulated information. In the two previous articles we discussed how to move that data fast. This third one is about how to protect it: encryption, integrity, access control, sovereignty and the threat that forces us to act today even though it looks distant, quantum computing.

Encryption at rest: SED, AES-XTS and the transition to FIPS 140-3

The first line of defence for any persistent data is encryption at rest. In enterprise storage the dominant pattern is the SED (Self-Encrypting Drive), a drive that implements the Trusted Computing Group’s Opal, Ruby and Enterprise standards and integrates a 256-bit AES-XTS encryption engine in the controller. The operational advantage is twofold: they encrypt at interface speed, with no perceptible performance penalty and without stealing CPU cycles, and they enable instant cryptographic erase.

The mechanism behind that erase is the two-tier key model of TCG Opal 2.0. Data is actually encrypted against a data encryption key (DEK) that never leaves the drive; that DEK, in turn, is protected by an authentication key derived from the user’s passphrase. Destroying or regenerating the DEK instantly renders terabytes of data unreadable without having to overwrite them: the cryptographic erase. For an organisation decommissioning 245 TB drives, this property is not a luxury, it is the only practical way to guarantee erasure.

SED hardware encryption and software encryption with AES-NI are not mutually exclusive, but they do have different profiles. Hardware acceleration keeps a measurable edge, on the order of 2 times on 4K random reads and writes compared with pure software, although the gap narrows on modern CPUs with AES-NI and ARMv8. The rule of thumb: SED for the baseline at no performance cost, and software encryption on top when you need key domain separation or end-to-end encryption.

The Achilles heel of encryption is never the algorithm, it is key management. Impeccable AES-256 protects nothing if the key is badly held. That is why serious storage encryption relies on an external key manager that speaks KMIP (Key Management Interoperability Protocol) to the arrays, separating custody of the key from the medium that encrypts. Keys are rotated periodically, backed up with their own availability guarantees (losing the key means losing the data as definitively as a fire would) and their access is audited. In regulated environments those keys live in certified hardware security modules (HSM). For an architect, the design decision is not “do we encrypt?”, the answer is always yes, but “who holds the keys and how are they governed?”, a question that, as we will see, sits at the heart of data sovereignty.

The point no architect should ignore in 2026 is the transition to FIPS 140-3. The CMVP stopped accepting new FIPS 140-2 validations in April 2022, and on 21 September 2026 it will move all remaining 140-2 certificates to the “historical” list. From that date, US federal agencies must not include historical modules in new procurements; existing systems can keep operating, but the compliance clock is running. And there is a logistical side effect: validation times have grown from 367 to 542 days, 42 % more, which strains vendor roadmaps. Anyone planning storage purchases on a multi-year horizon should verify 140-3 certification, not 140-2, for every cryptographic module. The reference framework for the overall design remains NIST SP 800-209, which covers encryption, isolation, authentication and authorisation for block, file and object storage.

Encryption in transit: the weak point of AI fabrics

Data is also exposed in motion, and here the high-performance fabrics of AI introduce specific tensions. For NVMe-oF over TCP there is in-band authentication, and for stronger protection the NVMe working group proposes IPsec over RoCEv2. Modern NICs (ConnectX-6 Dx, BlueField-2) secure NVMe-oF connections with hardware-accelerated IPsec and TLS, so encryption in transit does not throttle bandwidth.

The problem is InfiniBand. IPsec is not implemented over InfiniBand, since it is not an IP protocol, which leaves that fabric, ubiquitous in AI clusters, without one of the usual confidentiality tools. Nor does IPsec isolate well the RDMA connections of different users sharing the same interface. Academic research has documented attacks that exploit RDMA flaws in NVMe-oF storage applications. The conclusion for an architect is uncomfortable but important: in-transit security on an AI fabric is not inherited from the good habits of IP networks, and it demands explicit design of segmentation and isolation.

The threat arriving from the future: post-quantum cryptography

No storage security topic has changed its urgency as much as post-quantum cryptography (PQC), and the reason is an attack pattern that does not need a quantum computer to begin: "harvest now, decrypt later" (HNDL). Well-funded state actors are collecting encrypted data today with the intention of decrypting it once they have quantum capability. For any data whose confidentiality must last a decade, such as medical records, industrial secrets or proprietary training datasets, the threat is present, not future.

In August 2024 NIST finalised the first three PQC standards, and by mid-2026 they are the foundation on which the entire migration is built:

Standard	Algorithm	Function	Origin
FIPS 203	ML-KEM	Key encapsulation	CRYSTALS-Kyber
FIPS 204	ML-DSA	Digital signature (default)	CRYSTALS-Dilithium
FIPS 205	SLH-DSA	Backup signature (hash-based)	SPHINCS+

In March 2025 NIST additionally selected HQC, a code-based encapsulation mechanism, as a backup for ML-KEM with different mathematical assumptions. The NSA, for its part, set a demanding timetable in CNSA 2.0: by 2030 all software and firmware deployed in national security systems must use CNSA 2.0 signatures, with full enforcement expected around 2031-2033 and complete quantum resistance in 2035.

The pressure has intensified from the quantum hardware side. Between May 2025 and early 2026, several papers cut the estimates of qubits needed to break RSA-2048 from around 20 million to under a million, and even to figures on the order of 100,000 with new architectures. “Q-Day” around 2030 is increasingly cited by analysts, NIST and the NSA. It is not a certainty, but the safety margin has narrowed enough to make inaction imprudent.

The storage industry is already moving. NetApp announced PQC for data at rest in 2025, compliant with the NIST algorithms and integrated both at rest and in flight. Western Digital began integrating NIST-approved PQC into its Ultrastar drives in 2026, one of the first deployments in production infrastructure. Cohesity, Commvault and Quantum also have products along these lines. The dominant pattern in 2026 is not wholesale replacement but hybrid approaches that combine classical and post-quantum cryptography: they protect against a failure of the new algorithms and allow a gradual migration. For an architect, today’s concrete action is the cryptographic inventory, or crypto-agility: knowing which algorithms protect which data and planning their rotation.

Security specific to AI data: integrity and poisoning

Beyond classical encryption, AI brings a threat class of its own that falls directly on the data layer: data poisoning. An adversary manipulates training or fine-tuning data to corrupt what the model learns. Its distinctive trait is stealth: a poisoned model behaves normally for long periods before exhibiting the malicious behaviour, which makes it very hard to detect after the fact. The OWASP Top 10 for LLM applications of 2025 formally recognises “data and model poisoning” as an integrity attack category, with especially high risk when external sources are ingested.

The defence has moved towards provenance and lineage. Organisations are adopting rigorous documentation of every data source, a “digital chain of custody”, and the concept of the ML-BOM (Machine Learning Bill of Materials), analogous to the software SBOM. Frameworks such as the FINOS AI governance framework explicitly catalogue data poisoning among their risks. For storage, this means the system must be able to guarantee the integrity and immutability of training data and to record its lineage verifiably. Integrity stops being a desirable property and becomes a security control.

Model theft: when the asset to protect is the weights

There is one asset the traditional storage security conversation did not contemplate: model weights. Training a frontier model costs tens or hundreds of millions, and the result, a file of a few TB, concentrates all that value. Weight theft (model exfiltration) is consequently a first-order threat, and its attack surface is precisely the storage layer where checkpoints and final models live.

Protection combines several of the layers already described, applied with a specific focus. Encryption at rest with strict key control stops an attacker who reaches the drives from walking away with a usable model. Least-privilege access control limits who can read the weight directories, and auditing logs every access so anomalous exfiltration can be spotted; a process that suddenly reads 4 TB from a model directory is a signal. And confidential computing, which we will see further down, protects the weights even while they are being loaded into the GPU for inference. The lesson for the architect is that the trained model deserves, at minimum, the same defences as customer data, because its loss can be irreversible and its value greater.

Immutability and ransomware: the second phase of the attack

Ransomware remains the highest economic impact threat against storage, and its recent evolution points straight at backups. The reference defence is WORM immutability (Write-Once-Read-Many). In the object world, S3 Object Lock prevents modifying or deleting data even for an attacker with administrative credentials, neutralising the second phase of the attack, the encryption of the victim’s data. It has two modes: Governance, which can be overridden with specific permissions, and Compliance, which nobody can override, not even the root account. Object Lock creates a logical air gap that, for many use cases, makes tapes and physical air gaps redundant.

The numbers justify the investment. Ransom payments reached a record USD 1.1 billion in 2023; the average total cost of a ransomware attack was around USD 5.13 million in 2024, with an average recovery cost, excluding the ransom, of 1.53 million in 2025. Two out of three organisations suffered ransomware in the past year according to Sophos.

For an inference factory, ransomware has an angle of its own: the most valuable target is not only customer data but the model repository. Encrypting or deleting the weights the factory serves leaves it inoperative immediately, with a direct business impact, the service goes down, on top of the recovery cost. The defence is the same as for any critical asset, applied as a priority: immutable WORM copies of the model repository and of the vector indexes, so that a clean version of each model can always be restored even if the hot copy is compromised. A model that took months and millions to produce, or whose weights cannot be regenerated, deserves the strictest immutability protection available.

The storage-side response has been to integrate detection into the array itself. Dell PowerProtect Cyber Recovery offers an isolated, immutable vault with an automated air gap, and its CyberSense component uses AI and full content analysis, not just metadata, to detect ransomware corruption, with very high accuracy figures announced by the vendor. NetApp integrates Autonomous Ransomware Protection into the storage, creating immutable snapshots in real time by analysing data patterns at the storage layer. It is wise to treat the accuracy figures (99 % and above) as vendor claims rather than independent benchmarks, but the underlying trend is real: the storage array has stopped being a passive component and become an actor in the defence.

A useful framework for prioritising these investments is the annualised loss expectancy, which relates the impact of an incident to its frequency:

$$\text{ALE} = \text{SLE} \times \text{ARO}$$

where $ \text{SLE} $ is the expected loss per incident and $ \text{ARO} $ the expected annual frequency. Immutability and fast recovery act by reducing $ \text{SLE} $: even if the attack happens, the loss and the recovery time are bounded.

Access control and the cost of a breach

Encryption protects the data; access control decides who touches it. IBM’s Cost of a Data Breach 2025 report supplies the reference figure: the global average cost of a breach fell to USD 4.44 million, 9 % less and the first drop in five years, driven by faster containment thanks to AI-powered defences, with the average time to identify and contain falling to 241 days, the lowest in nine years. The United States, by contrast, set a record of 10.22 million. The largest cost component is still detection and escalation.

Mature architectures combine Zero Trust with RBAC as the base, ABAC attributes for fine grain and microsegmentation to prevent lateral movement inside the network. In an AI environment, where training data moves between data lake, pipeline and GPU cluster, microsegmentation and continuous contextual authorisation stop being optional.

Data sovereignty: the regulatory dimension

For a European company, AI storage security is inseparable from data sovereignty. The EU Data Act, applicable since 12 September 2025, mandates data portability and interoperability to eliminate vendor lock-in, and in its Chapter VII requires cloud providers operating in the EU to put in place technical, legal and organisational measures to prevent non-EU government access to non-personal data stored in the EU where such access would be unlawful under European law. This creates a documented tension with the US CLOUD Act, which empowers US authorities to claim data from providers under their jurisdiction regardless of where it is stored.

The technical answer gaining traction in 2026 is client-side encryption with full key ownership: whoever controls the keys controls the data, regardless of where it physically resides. “Whoever allows plaintext in the cloud cedes control” has become the motto of real sovereignty. For a European AI architect, data residency and key ownership are design decisions as important as performance.

The EU AI Act, furthermore, applies in phases that touch data governance: prohibited practices are enforceable since February 2025, obligations for general-purpose models since August 2025, and sanctioning powers since August 2026, although the provisional Digital Omnibus agreement of May 2026 postponed the deadline for Annex III high-risk systems to December 2027, a point worth re-checking given how recent it is.

Confidential computing: protecting data in use

Encryption at rest and in transit leaves a gap: data in use, decrypted in memory while it is processed. Confidential computing closes it with trusted execution environments (TEE). On the CPU, Intel TDX and AMD SEV-SNP launch confidential virtual machines with encrypted memory. The novelty for AI is on the GPU.

NVIDIA introduced confidential computing on the H100 (Hopper) with a CC mode and a root of trust in silicon: the CPU’s confidential VM exchanges encrypted data with an enclave in the GPU, extending the chain of trust from CPU to GPU. The Blackwell architecture (B200, GB200) has made the qualitative leap: it is the industry’s first GPU with TEE-I/O capability, with inline protection over NVLink and NVSwitch that removes the PCIe I/O bottlenecks of earlier generations. And what matters most for performance: thanks to AI-specific encryption engines and hardware-accelerated encrypted HBM access, the HGX B200 keeps its roughly 2 times advantage in training and 2.5 times in inference over the HGX H200 even with confidential computing fully enabled. This enables confidential training, inference and federated learning without the prohibitive penalty that previously made encrypting data in use unworkable, protecting both sensitive data and the model weights themselves from theft.

Security in a multi-tenant inference factory

Confidential computing takes on special meaning in an inference factory, above all if it serves several customers or teams on the same infrastructure. Here the attack surface is not only the disk at rest but the data in use of each request: the prompts, the context documents, the responses and, crucially, the KV cache. And the KV cache introduces a risk specific to inference that is worth naming: if a cache layer is shared between tenants to save compute, reusing prefixes as we saw in the performance article, badly isolated reuse can leak one customer’s content to another. The design rule is that prefix caching should only be shared within a single trust domain; between different tenants, KV cache isolation, whether in memory, in CXL or on NVMe, is a security control, not an optional optimisation.

The same principle extends to the factory’s other planes. The model repository must apply per-tenant access control so that nobody loads or extracts someone else’s model. The RAG vector database must segment indexes per customer, because an embedding retrieved from the wrong corpus is both a quality failure and a data leak. And telemetry, which records prompts and responses, is one of the most sensitive repositories in the whole installation, subject to the same encryption, retention and residency requirements as customer data. A well-designed inference factory treats each of these planes as an isolation domain, not as a resource shared for convenience.

An important technical nuance of confidential computing is attestation. A secure enclave is worthless if you cannot prove the data is really executing inside it and not in a compromised environment pretending to be one. Attestation is the cryptographic mechanism by which the hardware, the root of trust in the silicon of the GPU or the CPU, issues a verifiable proof of its state and identity before keys or data are entrusted to it. In a confidential AI architecture, the chain of trust extends from the CPU’s confidential VM to the GPU enclave, and every link is attested. For the architect, the operational question is who verifies those attestations and where the keys that are only released after a valid attestation are held: that is where confidential computing connects back to the key management from the start of the article.

A checklist for the architect

Gathering all of the above into actionable practice helps avoid leaving layers uncovered. At rest: SED with AES-XTS-256 as the baseline, external key management via KMIP with HSM, and verification of FIPS 140-3 certification on every module, anticipating September 2026. In transit: TLS 1.3 and IPsec wherever possible, with particular attention to isolation on InfiniBand fabrics, where the IP tools do not apply. Against the quantum threat: cryptographic inventory, crypto-agility and adoption of hybrid classical-PQC schemes for long-lived data, without waiting for Q-Day. Against ransomware: WORM immutability with S3 Object Lock in Compliance mode for critical copies, detection built into the array and tested recovery. On integrity: verifiable lineage and ML-BOM for training data, with reinforced protection of model weights. In the inference factory, additionally: isolation of the KV cache and of prefix caching between tenants, per-customer segmentation of the model repository and of the RAG vector indexes, and treatment of prompt and response telemetry as sensitive data. On governance: Zero Trust access control, microsegmentation and auditing. And on sovereignty: key ownership and data residency aligned with the EU Data Act and the EU AI Act. None of these layers substitutes for another; real security emerges from their combination.

Takeaways

AI storage security is a layered system that has to be designed all at once: encryption at rest with SED and migration to FIPS 140-3; encryption in transit with particular care on InfiniBand fabrics; crypto-agility and hybrid approaches to get ahead of quantum risk and harvest now, decrypt later; data integrity and lineage against poisoning; WORM immutability against ransomware; Zero Trust access control; data sovereignty through key ownership; and confidential computing to protect data and models in use. The good news is that the array is no longer passive: it defends. The bad news is that the most serious threat, the quantum one, demands action before it becomes visible.

The last article in the series closes with the property without which none of the previous ones matter: availability.

Sources

NIST, FIPS 140-3 Transition Effort — https://csrc.nist.gov/projects/fips-140-3-transition-effort
SafeLogic, What Happens on September 21, 2026 — https://www.safelogic.com/blog/what-happens-on-september-21-2026
NIST, Security Guidelines for Storage Infrastructure (SP 800-209) — https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-209.pdf
NIST, NIST Releases First 3 Finalized Post-Quantum Encryption Standards — https://www.nist.gov/news-events/news/2024/08/nist-releases-first-3-finalized-post-quantum-encryption-standards
PostQuantum.com, NSA Unveils CNSA 2.0 — https://postquantum.com/quantum-policy/nsa-cnsa-2-0-pqc/
Blocks & Files, NetApp boosts storage security with post-quantum encryption — https://www.blocksandfiles.com/security/2025/04/29/netapp-boosts-storage-security-with-post-quantum-encryption/1610355
The Quantum Insider, Western Digital Adds Post-Quantum Cryptography to Hard Drives — https://thequantuminsider.com/2026/05/19/western-digital-adds-post-quantum-cryptography-to-hard-drives/
FINOS AI Governance Framework, Data Poisoning (RI-9) — https://air-governance-framework.finos.org/risks/ri-9_data-poisoning.html
Object First, S3 Object Lock for Ransomware Protection — https://objectfirst.com/guides/immutability/s3-object-lock-for-ransomware-protection/
StorageReview, Dell enhances PowerProtect portfolio for cyber resilience — https://www.storagereview.com/news/dell-technologies-enhances-powerprotect-portfolio-for-improved-cyber-resilience
NetApp, Autonomous Ransomware Protection — https://www.netapp.com/cyber-resilience/autonomous-ransomware-protection/
IBM, Cost of a Data Breach Report 2025 — https://www.ibm.com/reports/data-breach
European Commission, Data Act — https://digital-strategy.ec.europa.eu/en/policies/data-act
artificialintelligenceact.eu, Implementation Timeline — https://artificialintelligenceact.eu/implementation-timeline/
NVIDIA, Confidential Computing on H100 GPUs — https://developer.nvidia.com/blog/confidential-computing-on-h100-gpus-for-secure-and-trustworthy-ai/
Corvex, Confidential Computing Meets NVIDIA HGX B200 — https://www.corvex.ai/blog/confidential-computing-meets-nvidia-hgxtm-b200-secure-ai-without-the-performance-trade-off

Storage in the AI era (4/4): availability

Mon, 22 Jun 2026 09:00:00 +0200

We have covered the state of the art, performance and security. The series closes with the property all the others depend on: availability. Storage that is blazingly fast and perfectly encrypted is worth nothing if a failure brings down an inference factory under SLA, halts a training run across thousands of GPUs or lets a petabyte-scale dataset corrupt itself in silence. And at AI scale, failures are not an exceptional event: they are the permanent operating regime. The difference lies in how you live with them: training resumes from a checkpoint; an inference factory, by contrast, cannot afford to stop, and must keep answering while a component fails underneath it. We will look at both cases, with the focus on continuous service.

Availability and durability are not the same thing

It is worth starting by separating two concepts that often get confused. Availability measures what fraction of the time the data is reachable; durability measures the probability of not losing it. A system can be very durable but poorly available (the data is safe but cannot be read right now) and vice versa.

Availability is modelled with the familiar relation between mean time between failures and mean time to repair:

$$A = \frac{\text{MTBF}}{\text{MTBF} + \text{MTTR}}$$

From that come the “nines” everyone quotes, whose translation into annual downtime is worth internalising:

Availability	“Nines”	Annual downtime
99.9 %	three nines	~8.8 hours
99.99 %	four nines	~52.6 minutes
99.999 %	five nines	~5.3 minutes
99.9999 %	six nines	~32 seconds

One detail that surprises anyone coming from the on-premise world: public cloud storage services do not usually offer an SLA above four nines (99.99 %). AWS S3, for instance, guarantees 99.99 % availability per region. Six nines are the territory of enterprise arrays in a stretch cluster configuration: Pure Storage offers them in that topology, against “five nines and a bit” on a single node. The formula makes it clear why reducing MTTR, recovering fast, is usually more cost-effective than chasing an impossible MTBF.

Durability: the “eleven nines” and what they hide

Durability is expressed in figures that look like marketing but have a precise meaning. The “eleven nines” of S3 (99.999999999 %) mean that, if you store ten million objects, you would expect to lose one every ten thousand years. How do you get there? AWS starts from the unrecoverable failure rates of the hardware, replicates across three independent availability zones per region and applies erasure coding with overlapping fragmentation. The honest caveat is that there is no agreed standard for how the nines are calculated: every vendor uses its own methodology, and comparing durability figures across providers is, strictly speaking, comparing assumptions.

The mechanism that produces those stratospheric numbers is the combination of redundancy and failure independence. If a system stores each object so that it tolerates the loss of several fragments, and distributes those fragments across independent failure domains (different drives, nodes, racks, availability zones), the probability of simultaneously losing all the fragments you need becomes astronomically small. The key word is independent: if the fragments share a single point of failure, the same power supply, the same switch, the same zone, the independence is fictitious and the calculated durability an illusion. A good share of real data losses in theoretically durable systems come from failure correlations the model did not contemplate: a batch of faulty drives from the same manufacturer, a software bug that corrupts every replica at once, an accidental deletion that propagates. Real durability demands diversity, not just quantity.

To anchor those assumptions in reality there is one invaluable source: Backblaze’s drive statistics. In its 2025 data (published in February 2026) the annualised failure rate (AFR) of its fleet of 344,196 drives fell to 1.36 %, down from 1.55 % in 2024, the lowest since 2022; the lifetime AFR remains stable at 1.30 %. Drives of 14-16 TB now make up 52 % of the fleet and those of 20 TB or more around 23 %. These numbers are the raw material from which any credible durability figure is calculated.

But total drive failure is not the most insidious threat. That would be silent corruption (bit rot). Enterprise drives have an unrecoverable read error rate on the order of one bit per $ 10^{14} $ bits read, roughly 11 TB, and that rate has barely improved while capacity has soared. A classic NetApp study of 1.53 million drives over 41 months found more than 400,000 checksum mismatches, of which 30,000 were not detected by the RAID controller and only appeared during scrubbing; nearline drives corrupt at ten times the rate of enterprise ones. The lesson: at petabyte scale, periodic scrubbing and end-to-end checksums (such as those of ZFS) are not optional, because the corruption RAID does not see is the one that destroys a training dataset without warning.

Redundancy: why classic RAID is no longer enough

The traditional way of tolerating drive failures, RAID with parity, has become problematic precisely because of capacity growth. The reason is rebuild time. Sequential write speed on drives has not grown at the pace of their capacity, so rebuilding a large drive takes longer and longer: a 16 TB nearline HDD in RAID 5 can take between 24 and 72 hours to rebuild. During that window the array is degraded and exposed to a second failure.

And there is a subtler risk. Rebuilding a drive forces you to read all the others in full, which multiplies the exposure to an unrecoverable read error. Rebuilding a RAID 5 of eight 8 TB drives means reading 56 TB; with enterprise drives that implies around a 30 % probability of hitting a URE that aborts the rebuild, and with consumer drives the probability approaches 96 %. That is why RAID 5 is, quite simply, deprecated for new arrays with drives larger than 4 TB. On top of that comes the write hole: if a power cut interrupts a write, data and parity can end up out of sync and corrupt the array. ZFS eliminates it with copy-on-write.

The modern answer has two pieces. The first is erasure coding (Reed-Solomon codes, $ k+m $ schemes): the object is split into $ k $ data fragments and $ m $ parity fragments, and any $ k $ of the $ k+m $ are enough to reconstruct it. Its efficiency against replication is notable. Storage overhead is:

$$O = \frac{k+m}{k}$$

A $ k=4 $, $ m=2 $ scheme gives an overhead of 1.5, so 1 TB logical occupies 1.5 TB physical and tolerates two simultaneous failures, against the factor of 3 of triple replication. At exabyte scale, that difference between 1.5 and 3 is decisive in cost. The price of erasure coding is more CPU on writes and rebuilds and more complex operations, which is why replication is still preferred for latency-sensitive data read locally.

The second piece is declustered RAID (dRAID): instead of concentrating the rebuild on a dedicated spare drive, it distributes rebuild I/O across all the drives in the pool. The effect on recovery time is drastic: a 20 TB drive that in traditional RAID would take more than 60 hours can resilver in under 15 because every surviving drive contributes at once. OpenZFS dRAID and IBM Storage Scale implement it. The underlying idea connects back to the availability formula: if you cannot avoid the failure, minimise MTTR by spreading the recovery work.

There is one variable that amplifies all of this and that flash has changed at the root: the speed of the medium during the rebuild. Rebuilding 10 TB on HDD, whose transfer rate is around 300 MB/s, can take more than 55 hours; on SSD, with transfers above 6,000 MB/s, it drops to under 3, twenty times faster. When flash is combined with dRAID, the vulnerability window after a failure shrinks from days to hours or minutes, and with it the probability of a second concurrent failure causing data loss. This is one of the underlying reasons why all-flash platforms have taken over in AI: they do not just perform better, they recover sooner, and at this scale recovery speed is data safety.

Replication and RPO/RTO: how much you can lose and how long you can wait

To tolerate the failure of an entire site, replication comes into play along with two metrics every continuity plan must set: the RPO (Recovery Point Objective, the maximum acceptable data loss, measured in time) and the RTO (Recovery Time Objective, the maximum acceptable downtime).

Synchronous replication mirrors each write to a second site before acknowledging it, guaranteeing an RPO of zero, zero data loss, but its reach is limited by latency: in practice it requires metropolitan links below about 10 ms, and it does not work for intercontinental distances. Asynchronous replication acknowledges the write locally and propagates afterwards, which allows any geographic distance in exchange for an RPO of minutes or hours. Active-active stretch cluster topologies (such as Pure ActiveCluster) achieve zero RPO and RTO within the metropolitan range, and some add a third asynchronous site to combine local consistency with geographic protection. The decision is always the same triangle: distance, cost and tolerable loss. You cannot optimise all three at once.

It is worth dismantling a myth about array high availability. People tend to assume that “active-active” is always superior to “active-passive”, but there is a performance nuance that matters. In an active-passive configuration, the secondary controller takes over the load after a failure without having been serving I/O, so post-failover performance is predictable. In active-active, both controllers serve load in normal operation; when one fails, the survivor has to absorb its own load plus the transferred one, and may miss the performance SLA exactly when it is needed most. There is no universal answer: the choice depends on whether you prioritise aggregate performance in normal operation or a performance guarantee in degraded mode. What is non-negotiable is non-disruptive upgrades (NDU): a serious AI array is upgraded with the system powered on and serving I/O, using the same dynamic failover routine it employs on a failure. If an upgrade requires stopping, that array is not up to a critical workload.

The permanent failure regime: why it matters to inference

This is where AI storage availability separates from everything above: at AI scale, hardware failures are not rare, they are continuous. The most quoted figure comes from Meta’s Llama 3 training on a cluster of 16,384 H100 GPUs: over 54 days it suffered 419 unexpected component failures, one failure every three hours on average, with a cluster MTBF of approximately 1.8 hours. More than 66 % of the interruptions were due to hardware faults: defective GPUs, HBM3 memory, SRAM, network switches and cables.

Although the figure comes from a training run, the underlying statistic applies equally to an inference factory: the same hardware fails at the same rate, and an inference fleet of hundreds or thousands of GPUs will see components drop daily. The difference lies in the response. Training absorbs the failure by resuming from a checkpoint; inference absorbs it with redundancy and hot switching, because it cannot stop. In both cases storage is the piece that makes recovery possible: in training, as the medium where the checkpoint lives; in inference, as the replicated repository from which a replica is stood up instantly to substitute the fallen one. That is why it pays to understand the checkpointing mechanism well even if the factory is an inference one: it is the same principle of “assume the failure and recover fast” taken to its most extreme form.

The reason is purely statistical. If an individual component has a given MTBF, the MTBF of a cluster of $ N $ identical components shrinks in inverse proportion:

$$\text{MTBF}_{\text{cl}} = \frac{\text{MTBF}_{\text{dev}}}{N}$$

Even if each GPU failed only once every six years (some 50,000 hours), a cluster of 100,000 GPUs would see a failure every half hour, and one of a million, every three minutes. This is the operational reality no AI storage design can ignore.

How do you live with this? With checkpointing, which is both a performance tool (we saw it in article 2) and the central resilience mechanism: after a failure, training resumes from the last checkpoint instead of starting from scratch, potentially saving weeks of compute. This inverts the usual relationship: storage is not only what has to be protected, it is the instrument that protects the GPU. There is even an underlying debate, with Epoch AI arguing that with checkpointing based on memory distributed across GPUs, rather than storage-based, hardware failures would stop limiting training size even beyond a million GPUs. Whatever the approach, the conclusion for the architect is that the availability of the checkpointing subsystem is the availability of the training run.

A revealing detail in Meta’s data is that more than two thirds of the interruptions were hardware (GPU, HBM3, SRAM, switches and cables), and that storage, when well designed, does not appear among the main causes of failure but among the recovery mechanisms. This inverts the usual psychology of the storage architect: at this scale the job is not so much about preventing your subsystem from failing, though it must, as about guaranteeing that, when anything else fails, training can resume quickly. The storage of an AI cluster is, to a large extent, resilience infrastructure for the rest of the cluster. Sizing the write bandwidth for checkpointing and the re-read speed after a restart is therefore an availability decision for the whole system, not an isolated array metric.

Availability in an inference factory: the service does not stop

Everything above (failures at scale, checkpointing, cluster MTBF) is described from the point of view of training, which is a batch job: if it goes down, it resumes and time is lost, but nobody is waiting on the other side. An inference factory is the opposite: an online service under an SLA, where every second of unavailability is a failed request and, often, a contractual penalty. Its availability is designed with a different mindset.

The first principle is that the service plane must survive the failure of any component without interrupting requests. That means redundant replicas of every served model, load balancing that automatically withdraws a fallen replica, and, in the storage plane, that none of the factory’s four planes is a single point of failure. The model repository must be replicated and available, because if it goes down you cannot start new replicas exactly when you need them to substitute the failing ones. The RAG vector database needs its own high availability, because without it the service degrades its quality or stops answering. And the KV cache layer, ephemeral as it is, conditions latency: losing a shared cache forces recomputation and spikes TTFT at once for every affected client.

The second principle is model updates with no downtime. In an inference factory models are rotated frequently, with new versions, adjustments and rollbacks, and doing so without cutting service demands progressive deployment patterns: blue-green (stand up the new version in parallel and switch traffic when it is warm) or canary (send a fraction of traffic to the new version and widen it if it behaves). Both patterns have a direct storage implication: during the transition two sets of weights coexist in the repository and, at times, in HBM, which demands capacity and load bandwidth so the new version is ready before it receives traffic. A slow model repository does not just delay autoscaling: it slows every deployment and lengthens the risk window.

The third principle is elastic autoscaling, which turns cold start time, the time to load weights from storage that we saw in the performance article, into an availability metric. If starting a new replica takes minutes because the repository is slow, the factory does not absorb the peaks and degrades or rejects requests precisely when demand is highest. The real elasticity of an inference factory is limited by the speed at which its storage delivers models. That is why many installations keep a reserve of weights in hot local flash and use accelerated load paths: availability under variable load is won, in large part, in the storage layer.

The fourth principle is redundancy of the service plane itself, which can be sized with numbers. If a service replica has an individual availability $ a $ (limited by its GPU, its node and its access to storage), a service with $ n $ independent replicas in which one is enough to answer reaches an availability of:

$$A = 1 - (1 - a)^{n}$$

The formula explains why redundancy is so effective: with reasonably available replicas, each additional replica adds nines to the service. But its hypothesis is the same one we saw in durability: independence. If every replica depends on the same model repository, the same vector database or the same KV cache layer, that shared component is the real limit on availability no matter how many GPU replicas you add. Hence, in an inference factory, replicating and giving high availability to the storage planes (repository, vector DB, cache) is as important as replicating the serving GPUs themselves: those, and not the compute, are the single points of failure that usually get overlooked.

The conclusion is that in an inference factory the availability of storage is measured in terms of continuous service: not “how long do I take to recover the data?” but “does the service keep answering while a component goes down, a model is updated or demand doubles?”. It is a target of five or six nines over a living system, not durability over an archive at rest.

Disaster recovery: the 3-2-1 rule and the petabyte problem

For the major disaster, loss of a site or a destructive attack, the reference is still the 3-2-1 rule: three copies of the data, on two different media types, with at least one off site. Its 2025 evolution adds two requirements: immutability (the off-site copy must be unalterable, connecting with the anti-ransomware defence of the security article) and verified recoverability (a backup you have not tested restoring is not a backup).

At AI scale a physical obstacle appears that is hard to dodge: the recovery time of massive datasets. Restoring petabytes from an off-site copy can take days or weeks over standard connections, an unacceptable RTO for many operations. Organisations with petabytes cannot even do the initial backup to cloud over conventional internet; they need physical transfer or dedicated links. Sizing the recovery, not just the copy, is where most AI DR plans fail.

When the cloud goes down too

It is worth remembering that delegating to a hyperscaler does not eliminate the problem, it only moves it. 2025 left a forceful reminder: the big AWS outage of 19-20 October 2025, caused by a race condition in an internal DynamoDB microservice that generated an empty DNS record, cascaded to EC2 and lasted some fifteen hours, with more than seventeen million reports and global services down. There were precedents in July 2024 (us-east-1, via Kinesis) and February 2025 (Stockholm). The moral for a critical AI architecture is the usual one: multi-region or multi-cloud resilience is a conscious design decision, not a property you inherit by signing up for cloud.

The trend: storage that heals itself

The evolution vector for 2026 is the incorporation of AI into the storage layer itself for self-healing defence: predicting failures before they happen, automatic tiering and migration, pre-emptive remediation and instant recovery from ransomware. Storage hardware “is no longer passive”, as we saw in security. At exabyte scale, object storage is consolidating as the core because it combines massive durability with flexible metadata and versioning, while tape re-emerges as an active archive tier for AI datasets, fighting the cost of flash and HDD. With data growth of 30-40 % a year, automating resilience stops being a convenience and becomes a necessity: at that scale no human team manages failures by hand.

Commercial guarantees have moved in the same direction. Some vendors already offer contractual SLAs that were previously unthinkable: Pure, with its Evergreen//One model, announces up to ten distinct guaranteed SLAs, including six nines of uptime, zero data loss, a capacity buffer and zero planned downtime for upgrades, plus a specific post-cyberattack recovery SLA that guarantees clean arrays. Whatever the vendor, the trend signals a change of model: from selling hardware to guaranteeing availability outcomes. For the architect, this shifts part of the risk onto the provider, but it does not exempt you from designing: a contractual SLA compensates financially for an outage, it does not prevent it, and the continuity of a training run worth millions of euros is not restored with a credit note.

Availability is governed with observability

None of these techniques works blind. Sustaining availability at AI scale demands observability: monitoring the state of every drive (with SMART-style telemetry and failure prediction), the progress and integrity of scrubs, replication latency against its RPO target, and the health of checkpoints, which are written, yes, but which also have to be re-readable. The most deceptive metric is the backup that has never been restored: an unverified copy is a hypothesis, not a guarantee. The operational discipline that distinguishes resilient infrastructure is the periodic recovery test, at realistic scale, measuring real RTO against the target. At the scale of petabytes and thousands of GPUs, what is not measured cannot be guaranteed, and what is not rehearsed tends to fail precisely on the day it is needed.

Takeaways, and closing the series

The availability of AI storage is designed on four pillars: efficient redundancy (erasure coding and dRAID instead of classic RAID, which no longer copes with large drives), verified durability (with scrubbing and checksums against silent corruption), replication with explicit RPO/RTO according to criticality, and a pipeline resilience that assumes failure as the normal state and leans on checkpointing. The availability formula is a reminder of the practical priority: when you cannot avoid the failure, and at AI scale you cannot, what makes the difference is recovering fast.

With this we close the series. The state of the art gave us the map; performance, the reason for being; security, the protection; and availability, the guarantee that everything above will still be there tomorrow. In the AI era, storage has stopped being the silent basement of the datacenter and become what decides whether the GPUs, the most expensive and scarcest resource, work or wait.

Sources

ActualTech Media, RAID5 and Large Disks: Dealing with Rebuild Times — https://www.actualtechmedia.com/io/raid-disk-rebuild-times/
DiskInternals, RAID Rebuild Time and URE risk — https://www.diskinternals.com/raid-recovery/raid-rebuild-time/
Tech Buzz Online, Erasure Coding vs Replication — https://techbuzzonline.com/erasure-coding-vs-replication-storage-systems/
simplyblock, Declustered RAID (dRAID) — https://simplyblock.io/glossary/draid/
The Register, Data durability statements (11 nines) — https://www.theregister.com/2018/07/19/data_durability_statements/
Backblaze, Drive Stats for 2025 — https://www.backblaze.com/blog/backblaze-drive-stats-for-2025/
StorageReview, Backblaze 2025 Year-End Drive Stats — https://www.storagereview.com/news/backblaze-2025-year-end-drive-stats-annual-afr-falls-to-1-36-as-high-capacity-drives-dominate-fleet
Backup Central, Does Undetectable Bit Error Rate Matter? — https://backupcentral.com/does-undetectable-bit-error-rate-matter/
Pure Storage / Everpure, The Storage Reliability Imperative — https://blog.everpuredata.com/perspectives/data-storage-reliability/
Pure Storage, ActiveCluster — https://www.purestorage.com/products/purity/activecluster.html/
Tom’s Hardware, Faulty H100 GPUs and HBM3 — one failure every three hours (Llama 3) — https://www.tomshardware.com/tech-industry/artificial-intelligence/faulty-nvidia-h100-gpus-and-hbm3-memory-caused-half-of-the-failures-during-llama-3-training-one-failure-every-three-hours-for-metas-16384-gpu-training-cluster
Epoch AI, Hardware failures won’t limit AI scaling — https://epoch.ai/blog/hardware-failures-wont-limit-ai-scaling
Acronis, What is the 3-2-1 Backup Strategy (2025) — https://www.acronis.com/en/blog/posts/backup-rule/
InfoQ, Race Condition in DynamoDB DNS System (AWS Oct 2025 outage) — https://www.infoq.com/news/2025/11/aws-dynamodb-outage-postmortem/
TechTarget, Top data storage trends 2026 — https://www.techtarget.com/searchstorage/tip/Top-data-storage-trends

AI Infrastructure on lo0 — Tech Blog

Storage in the AI era (1/4): the state of the art

The underlying problem: the memory wall

HBM4: the ceiling rises, but it stays expensive

NAND and SSD: capacity explodes, the interface widens

CXL: from PowerPoint to production

Parallel file systems: the GPU battlefield

The rack as the unit of compute: GB200, Rubin and BlueField-4

The inference stack: when the SSD joins the memory hierarchy

Anatomy of storage in an inference factory

The other end: object, lakehouse and vectors

The data lifecycle: tiering as an architectural decision

The market context: a supply supercrisis

What this changes for the architect

Takeaways

See also

Sources

Storage in the AI era (2/4): performance

The metric that really matters: accelerator utilisation

MLPerf Storage: the benchmark that does measure the right thing

The numbers from commercial platforms

The I/O pattern of training and GPUDirect Storage

The real cost of checkpointing (when there is training)

Performance in an inference factory

Prefill, decode and their disaggregation

The KV-cache as a performance layer

Model load time and cold start

RAG performance and vector search

How to measure your own: beyond the spec sheet

The cost of getting it wrong

The physical layer: networks and PCIe Gen6

Three questions before sizing

Takeaways

See also

Sources

Storage in the AI era (3/4): security

Encryption at rest: SED, AES-XTS and the transition to FIPS 140-3

Encryption in transit: the weak point of AI fabrics

The threat arriving from the future: post-quantum cryptography

Security specific to AI data: integrity and poisoning

Model theft: when the asset to protect is the weights

Immutability and ransomware: the second phase of the attack

Access control and the cost of a breach

Data sovereignty: the regulatory dimension

Confidential computing: protecting data in use

Security in a multi-tenant inference factory

A checklist for the architect

Takeaways

See also

Sources

Storage in the AI era (4/4): availability

Availability and durability are not the same thing

Durability: the “eleven nines” and what they hide

Redundancy: why classic RAID is no longer enough

Replication and RPO/RTO: how much you can lose and how long you can wait

The permanent failure regime: why it matters to inference

Availability in an inference factory: the service does not stop

Disaster recovery: the 3-2-1 rule and the petabyte problem

When the cloud goes down too

The trend: storage that heals itself

Availability is governed with observability

Takeaways, and closing the series

See also

Sources