Kubeflow in depth: which pieces are worth it in an on-premise LLM platform and which ones you already have

· By José Manuel Román
Contents

Second post in the platform and self-service run, after Backstage and before Knative. The question here is not what Kubeflow is, but which parts of Kubeflow still make sense once you have built half a platform on your own.

TL;DR

Kubeflow graduated in the CNCF on 17 August 2026, and that headline hides the important one: it is no longer a monolithic platform, it is an umbrella of independent subprojects, each with its own repository, its own version and its own life. That changes how you have to evaluate it entirely. The 2020 question, should I install Kubeflow?, has become the 2026 one, which piece of Kubeflow do I install, and at what cost?

For a platform that already has RKE2, vLLM, KServe, Flux and Volcano or Kueue, the honest answer is that Kubeflow as a whole duplicates almost everything you already have. KServe no longer lives under Kubeflow: it left, and since November 2025 it has been a CNCF project in its own right, so anyone serving models with KServe does not need Kubeflow to serve. The Trainer leans on Volcano and Kueue, it does not replace them. And the full manifest drags in Istio, Dex and cert-manager as hard dependencies that clash with what you already operate. The only thing that may be worth installing on its own is Pipelines, as an orchestrator for data and training flows, and perhaps the Trainer for its fine-tuning templates. The rest you either already have, or do not need.

The analogy: the hypermarket and the specialist shop

Kubeflow was born as a hypermarket: a single building with everything an ML team might need, from notebooks to the inference server, taking in pipelines and hyperparameter tuning along the way. It made sense in 2019, when assembling each piece separately was a project in itself and there were no standards.

By 2026 the hypermarket has been converted into a street of specialist shops. The butcher’s counter, KServe, went independent, opened its own premises and is doing better alone. The fishmonger’s, the Trainer, now depends on the central market next door, Volcano and Kueue, instead of having its own cold store. And the shared building still demands that you put in its wiring, its plumbing and its security system (Istio, cert-manager, Dex) even though you already have your own.

Anyone arriving today with a half-full fridge, which is the case for anyone with a platform already running, does not need the whole hypermarket. They need to know which shop to walk into for what they are missing, and to avoid buying twice what they already have at home.

What Kubeflow is in 2026

Kubeflow was created by Google in 2017 and graduated in the CNCF on 17 August 2026, reaching the top level alongside Kubernetes or Prometheus. The graduation figures are serious: more than 6,600 contributors from over a thousand organisations, and adopters such as Bloomberg, NVIDIA, Red Hat or Spotify.

The structural change is the one that matters for the decision. The full distribution, the Kubeflow Community Distribution, is numbered by year and month (the current one is 26.03, from March 2026, on a six-monthly cadence), but each component is by now a subproject with its own repository and its own cycle, usable independently. The distribution only packages them together. That means “installing Kubeflow” is no longer an atomic decision: it is a shopping list.

Component by component

Kubeflow Pipelines: the piece you may actually want

Pipelines orchestrates containerised ML workflows. They are written in Python with its SDK, which compiles to an intermediate YAML representation so they are portable, and underneath they run on Argo Workflows. It is the piece most likely to add value to an existing platform: a flow of “ingest, embeddings, fine-tuning, evaluation, deployment” is exactly what you need to operate RAG and adapters repeatably.

The small print has two parts. The first is that if you already use plain Argo Workflows, Pipelines adds a layer of SDK, interface and metadata on top of the same engine: it does not bring a new scheduler, it brings the typed component model and the runs interface. The second is that its metadata and lineage system, MLMD, is being removed from Pipelines itself, so it is not wise to build the platform’s traceability on that particular piece.

Kubeflow Trainer: it leans on your scheduler, it does not replace it

The old Training Operator, the one with PyTorchJob and TFJob, has been rewritten as Kubeflow Trainer, with a single TrainJob API that unifies all frameworks. Version 2.2 positions itself explicitly for distributed training and LLM fine-tuning, with support for PyTorch, DeepSpeed, HuggingFace and company.

A common expectation needs correcting here. The Trainer does not replace Volcano or Kueue: it leans on them. The TrainJob carries a podGroupPolicy field that automatically creates the Volcano PodGroup objects for gang scheduling, and it integrates with Kueue’s queues. If you already built that scheduler, as we saw in its post, the Trainer adds the TrainJob abstraction and some fine-tuning templates on top of what you already have, not a new scheduling layer. It may be worth it for the templates; not for the scheduler.

Katib: only half relevant

Katib does Kubernetes-native AutoML: hyperparameter optimisation with algorithms such as Bayesian optimisation, TPE or Hyperband, plus architecture search. In the LLM era it has repositioned itself, with official documentation for tuning fine-tuning hyperparameters and even for tuning RAG pipelines.

The practical reading is lukewarm. Architecture search is irrelevant for anyone doing LoRA and RAG. Hyperparameter optimisation can be useful for learning-rate or LoRA-rank sweeps, but it competes with doing it from the SDK of the QLoRA runbook itself or with Ray Tune, and it rarely justifies dragging in Katib for that alone.

KServe: the piece that is no longer in Kubeflow

This is the fact that settles half the evaluation. KServe left Kubeflow. It was born inside the project in 2019, was donated to the Linux Foundation in 2022, was renamed from KFServing to KServe and graduated from Kubeflow that same year, and since November 2025 it has been a CNCF incubating project in its own right. In the 2026 taxonomy it appears as an ecosystem project, external, not as a core component.

The consequence is direct: anyone already serving with KServe does not need Kubeflow to serve. And KServe is exactly where the LLM action is: its 0.15 release strengthened the vLLM backend and introduced an LLMInferenceService resource with disaggregated serving, prefix caching, per-variant autoscaling and OpenAI-compatible APIs. If your serving layer is vLLM on KServe, you already have that layer complete and Kubeflow adds nothing to it.

Model Registry, Notebooks and the rest

The Model Registry, now reorganised under the name Kubeflow Hub, is an index of models, versions and metadata. It stores pointers and states, not the bytes. Against MLflow, it lacks the experiment and metric tracking that is MLflow’s strength, to the point that the guides recommend combining them rather than substituting one for the other; and against an OCI registry, it does not package the binary, which still lives wherever you put it. It is still a young component, below version 1.0.

The Notebooks live in two versions, the stable 1 and a 2 redesigned on CRDs that is still in alpha. The Spark Operator remains in the core; Feast, by contrast, has left the core and is now an ecosystem project, a nuance worth being clear about before counting on it.

Multi-tenancy: powerful and tied to Istio

Isolation between teams is done with Profiles, a CRD that wraps a namespace and gives it RoleBindings, ServiceAccounts and Istio authorisation policies that validate an identity header derived from OIDC. It is a complete model, but it has an Achilles heel: it depends entirely on the Istio sidecar and on that header. If traffic bypasses the mesh or someone forges the header, isolation breaks, and all multi-tenant security ends up coupled to Istio, which clashes head-on if you already operate another mesh or no service mesh at all.

The cost of installing the whole thing

Here is the main reason not to install the full distribution on top of an existing platform. The 26.03 manifest drags in as hard dependencies: Istio, cert-manager, Dex for OIDC, OAuth2-Proxy, and Knative Serving and Eventing for KServe. It recommends 16 GB of RAM and 8 vCPU as a minimum, and the aggregate of all components comes to around 4.4 CPU cores and 12 GB of memory for the control plane alone.

The project itself acknowledges the problem. There is an open thread in its repository, bluntly titled “community feedback on Kubeflow’s complexity”, where it is admitted that Kubeflow practically forces a dedicated cluster because of the assumptions it makes about what is installed, that Istio and Dex should be swappable and not fixed dependencies, and that a lighter Helm that installs only what is needed is missing. For anyone already operating cert-manager, an ingress and GitOps with Flux, the full manifest does not coexist: it duplicates and tramples the base infrastructure.

Kubeflow against the alternatives

The field is wide and almost all the comparative literature is written by vendors with a product of their own, so it has to be read with tweezers. With that caution:

Plain Argo Workflows is the engine Pipelines uses underneath; if all you need is container graphs, Pipelines is overhead. MLflow is strong in experiment tracking and lightweight registry, installs in minutes, and does not orchestrate distributed training: it is complementary, and in fact we use it in the post on prompt versioning. Flyte is a typed orchestrator that many perceive as lighter to operate than full Kubeflow. And Ray on Kubernetes is the most serious competitor for LLM workloads, because with a single runtime it covers training, serving and tuning, overlapping at once with the Trainer, Katib and KServe.

Where Kubeflow wins is in breadth and in governance: a CNCF-graduated umbrella with adoption by large companies. Where it loses is in operational weight, in hard dependencies, and in the fact that its best pieces are already independent projects that do not require the rest.

The real fit with LLM workloads

The project has moved towards GenAI, and that has to be acknowledged: there is an SDK with LLM fine-tuning templates, the Trainer adds support for reinforcement post-training methods, and new agent-oriented pieces have appeared. It is not empty marketing, there is real work there.

But the question for a specific platform is not whether Kubeflow does LLM: it is which part I have not already duplicated. On a stack of RKE2, vLLM, KServe, Flux and Volcano or Kueue, the balance looks like this:

ComponentDoes it add anything new to your stack?
KServeNo: you already have it, and it is no longer Kubeflow’s
TrainerOnly the fine-tuning templates; you already have the scheduler
KatibMarginal: LoRA HP sweeps, little more
PipelinesPossibly yes, as a flow orchestrator, but it runs on Argo
Model RegistryOptional: competes with MLflow and with your OCI registry
NotebooksDepends on what you use for notebooks today
Multi-tenancyNo, if you already isolate by namespace without tying everything to Istio

Operational traps and honest scepticism

Do not install the full manifest on a platform that already works. You are going to duplicate Istio, cert-manager and the rest, and fight a dependency war with no prize at the end. If something in Kubeflow interests you, install it as a standalone subproject.

KServe is not Kubeflow. It is the most common category error of 2026. If someone justifies building all of Kubeflow “to serve models”, you already know they have not checked that KServe left long ago and lives on its own.

The Trainer does not save you the scheduler. It needs Volcano or Kueue underneath. If you expected it to solve GPU gang scheduling by itself, it does not.

Multi-tenancy ties you to Istio. Before adopting Profiles, ask yourself whether you want security between teams to depend on one specific mesh. If you already isolate well by namespace and RBAC, you may not need that layer.

MLMD is on its way out. Do not build your pipelines’ traceability on the metadata system the project itself is retiring.

For an inference factory

The conclusion is comfortable to apply. Kubeflow in 2026 is not a yes-or-no decision, it is a menu you order from à la carte. On a platform that already serves with KServe, schedules with Volcano and deploys with Flux, the menu shrinks a great deal.

If you need to orchestrate data and training flows repeatably, and you do not want to build plain Argo Workflows with its ergonomics, Pipelines is the piece worth evaluating, installed standalone. If you do distributed fine-tuning frequently and want a uniform abstraction on top of your scheduler, the Trainer may save you some templates, knowing that it leans on the Volcano you already have. Everything else you either have, or a specialist piece covers better.

The expensive mistake would be installing the whole hypermarket to buy bread. Kubeflow stopped being that; treat it as what it is, a street of shops, and walk only into the one you are missing. The next piece in this run, Knative, is precisely one of those specialist shops, the one that decides whether switching off the GPU when nobody is using it pays off or costs you dear.

See also

Sources