Accelerate innovation across the DSX ecosystem and bring AI factory solutions to market faster with the open foundation for AI workload and GPU orchestration.
Today, NVIDIA is making the full NVIDIA Run:ai platform available as open source software, giving customers, developers, and ecosystem partners access to production-grade AI workload and GPU orchestration capabilities. This release expands the NVIDIA DSX AI Factory Platform by establishing NVIDIA Run:ai as an open foundation that enables the broader ecosystem to bring differentiated solutions to market faster in response to evolving customer needs.
Over the past several years, NVIDIA Run:ai has helped organizations run AI workloads on Kubernetes at scale by dynamically allocating GPU resources and managing workloads across the full lifecycle, from data processing and training to inference.
By open sourcing NVIDIA Run:ai platform capabilities as part of NVIDIA DSX AI Factory, it is now possible for every NVIDIA DSX ecosystem partner and AI cloud to increase workload throughput, improve GPU utilization, and operate shared AI infrastructure more efficiently and effectively.
This post details the open source release, which includes the complete NVIDIA Run:ai platform and four components that can also be adopted independently: KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, Workload Map. Together with additional platform code, these components form the integrated NVIDIA Run:ai platform. Platform teams and ecosystem partners can adopt NVIDIA Run:ai as an integrated whole, build on it for their own products and services, or integrate individual components based on their requirements.
NVIDIA Run:ai can be adopted directly as open-source software or through NVIDIA DSX ecosystem partner offerings.
The complete source code for NVIDIA Run:ai is available in the NVIDIA Run:ai repository within the NVIDIA DSX AI Factory GitHub organization. Teams can inspect the code, contribute to its development, build on top of the platform or adopt its modular components independently.
The complete open source NVIDIA Run:ai platform is also available on NVIDIA NGC as a Community Product, with a Helm chart for installing the integrated stack.
Note: The NVIDIA Run:ai source code is governed by its applicable open source licenses. Downloading the Community Product from NGC requires acceptance of the applicable NVIDIA Software License Agreement and Product-Specific Terms for NVIDIA AI Products. No paid NVIDIA Run:ai subscription or NVIDIA support entitlement is required for the Community Product, and NVIDIA enterprise support is not included.
Open source NVIDIA Run:ai gives NVIDIA DSX ecosystem partners direct access to AI workload and GPU orchestration innovations they can use in their own NVIDIA DSX AI factory solutions. By adopting the complete platform or integrating individual components, partners can bring those innovations to market faster and respond more quickly to evolving customer and operational requirements.
For NVIDIA Enterprise AI Factory deployments, NVIDIA Run:ai continues to be available through NVIDIA-supported commercial offerings including as a standalone product, through NVIDIA Mission Control and NVIDIA AI Enterprise, and select cloud marketplaces. Availability, licensing, and support terms vary by offering.
For teams that already have a platform and need to fill a specific capability gap, the following sections describe when and how to use each modular component independently.
This section details the four components included in the open source release: KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, and Workload Map. With additional platform code, these components form the integrated NVIDIA Run:ai platform. You can adopt NVIDIA Run:ai as an integrated whole or integrate individual components based on your unique requirements.
KAI Scheduler is the Kubernetes-native scheduler that powers the NVIDIA Run:ai platform. NVIDIA released KAI Scheduler as open source software under the Apache 2.0 license and contributed it to CNCF as a Sandbox project. KAI remains the upstream engine behind NVIDIA Run:ai, and all scheduler development happens in the community repository, so every improvement flows through the same codebase. NVIDIA continues to support the project’s maintainers and work with the broader ecosystem to advance AI workload scheduling at scale.
KAI Scheduler is purpose-built for GPU workloads at scale, coordinating:
To learn more about key features, visit the kai-scheduler/KAI-Scheduler GitHub repo.
Use KAI Scheduler for diverse AI workloads that require fair access to cluster resources and topology-aware placement. It can be deployed in an existing cluster alongside the current software stack without requiring the complete NVIDIA Run:ai platform.
Follow the project, ask questions in the #kai-scheduler channel on CNCF Slack, and contribute on the kai-scheduler/KAI-Scheduler GitHub repo.
The NVIDIA Run:ai platform has long supported fractional GPU allocation, enabling multiple workloads to share a GPU with hard memory isolation. Fractions can be requested on demand and set up on the fly, with no per-node setup from an administrator in advance. It is one of the most widely used capabilities among ML teams running workloads that do not require the full memory of a GPU, including interactive notebooks and inference for smaller models. NVIDIA Run:ai pioneered that experience with a proprietary implementation.
GPU Fractioning Operator brings CUDA-native GPU fractioning to Kubernetes and the proprietary implementation has been retired in favor of the native capability available in CUDA 13.4. The CUDA driver enforces a hard limit on GPU memory for each workload, and each workload sees only the GPU memory allocated to its fraction. What is still needed is a clean way to consume it on Kubernetes. This is what GPU Fractioning Operator provides: an operator that enables CUDA GPU fractions on a cluster, along with a metrics exporter for per-pod visibility into fractions in use. It is a Kubernetes integration layer that allows workloads to use CUDA-native fractioning, and it is scheduler-agnostic.
Use the GPU Fractioning Operator for workloads that can share underutilized GPU capacity, including notebooks, development sessions, and smaller models. GPU Fractioning Operator integrates with KAI Scheduler to translate available GPU capacity into an actual placement decision, while remaining compatible with other schedulers.
GPU Fractioning Operator brings CUDA 13.4-native GPU fractioning to Kubernetes and integrates with KAI Scheduler. To get started, refer to the GPU Fractioning Operator documentation.
While the scheduler decides how work is placed, KAI Resource Management governs who gets what:
Together, these let platform teams move beyond raw Kubernetes scheduling and manage GPUs the way organizations actually operate: by team, by department, by initiative, and by resource guarantee.
Use KAI Resource Management for heterogeneous clusters shared by multiple teams that require defined shares of capacity. For example, a platform team could guarantee 50% of the cluster to research while preventing any one team from monopolizing the remaining capacity. If you are already tracking quota entitlements in a spreadsheet or hand-maintaining namespaces per team, this is the layer that replaces it. It can be combined with KAI Scheduler in your clusters.
Read more about how these concepts map to real deployments in the KAI Resource Management documentation.
Workload Map is the contract NVIDIA Run:ai uses to support any workload type. KAI Scheduler uses it for scheduling and gang semantics; the rest of the platform uses it for workload discovery, monitoring and lifecycle management.
In Kubernetes, an AI workload is almost never a bare Deployment or Job. It is a PyTorchJob, a RayCluster, a JobSet, a KServe InferenceService, a Dynamo graph deployment, or a company’s own custom resource. Each type has its own internal structure, components with different roles (driver and workers, head and workers, prefill and decode), its own scaling and gang semantics, and its own way of reporting status.
Workload Map provides a consistent way to treat these workloads as first-class objects throughout their lifecycle:
This workload-management layer sits between Kubernetes primitives and the schedulers, dashboards, SRE and troubleshooting tools, cost tools, backup tools, and ML platforms that serve users. Instead of each tool building a private, partial integration for every CRD, Workload Map defines each workload type once so any platform or tool can schedule, observe, act on, and govern it consistently without hardcoding each CRD.
Use Workload Map for platforms and tools that must understand and manage multiple Kubernetes workload types without hardcoding each CRD. It ships as a Go library that works offline on unstructured objects, so you can adopt it without adding anything to your cluster. It is deliberately narrow: Workload Map describes workload structure and does not schedule, queue, autoscale, or propagate across clusters.
Learn more in the dsx-ai-factory/workload-map GitHub repo and check out the adopter projects.
The following open source projects address complementary AI infrastructure challenges. They can be adopted independently or incorporated into broader solutions that include the complete NVIDIA Run:ai platform or selected components.
Cold-starting an inference workload on Kubernetes can take several minutes, during which time GPUs are allocated but idle for a single step: loading model weights from storage onto the GPU. Model Streamer reads weights concurrently and streams them directly into GPU memory, cutting load time for large models.
Since its release, Model Streamer has been adopted across the inference stack, including by inference engines, cloud providers and storage platform vendors.
Model Streamer is integrated upstream in vLLM and SGLang, where it can be enabled with the –load-format runai_streamer flag, as well as in ModelExpress. Support covers local filesystems, S3-compatible stores, and network file shares alongside several cloud backends detailed below.
Major cloud providers now document Model Streamer in their own guidance.
Several AI data infrastructure partners have independently benchmarked Model Streamer and published their results:
Use Model Streamer when starting new replicas and loading model weights is the primary bottleneck.
Emerging workload patterns—including agentic pipelines, disaggregated serving and mixture-of-experts training—consist of multiple connected components that must scale together. Grove provides a Kubernetes API for orchestrating these workloads, from a single pod to deployments spanning tens of thousands of GPUs.
Grove provides topology-aware autoscaling and hierarchical gang scheduling, allowing components such as prefill and decode to start as all-or-nothing units. On NVIDIA GB200 NVL72, it also manages ComputeDomain resources automatically, enabling multinode NVLink performance through a single annotation instead of hand-authored YAML.
Grove is a modular part of NVIDIA Dynamo that can be used independently, and it already powers training and inference pipelines at scale across a range of companies. Recent work includes support for llm-d and Volcano as a scheduler backend, with Kueue support in progress.
Use Grove for graph-based workloads with distinct roles and coupled scaling requirements.
NVIDIA Run:ai is available now as open source software in the NVIDIA DSX AI Factory GitHub organization. To learn more and get started, check out these resources:
NVIDIA Run:ai full platform
Related open source projects include:
Model Streamer for fast model weight loading
We’re excited to bring this work to the community and to build the next generation of AI workload and GPU orchestration in the open. We welcome your contributions, issues, and feedback.
In less than a year, Pluto has reached multi-million-dollar ARR, secured more than 1 million…
The OV Spark and OV Spark Pro combine image acquisition, model training, inference, PLC communication…
The transaction adds integrated-voltage-regulator and vertical-power-delivery technology to Navitas’ GaN and SiC power-semiconductor portfolio. Navitas…
The new enhancement-mode GaN transistors combine low on-resistance, compact FCLGA packaging and options for dual-side…
The Simplicity AI SDK connects AI coding assistants to device configuration, debugging and target-hardware analysis…
Artificial intelligence workloads are redefining the power requirements of modern data centers. Increasing GPU performance…