NVIDIA Groq 3 LPX Enters Full Production for Ultra-Fast Agentic AI Inference

NVIDIA has announced that its Groq 3 LPX interactive AI inference accelerator is now in full production, extending the company’s Vera Rubin platform with hardware specifically designed for ultra-low-latency token generation in agentic AI workloads. The announcement was made at Hot Chips 2026.

Agentic AI systems can generate very large volumes of tokens across hundreds or thousands of inference steps, placing different demands on compute infrastructure than conventional chatbot workloads. NVIDIA says Groq 3 LPX is designed to accelerate the generation phase of inference, complementing Vera Rubin NVL72 systems by increasing token-generation rates for latency-sensitive and context-heavy applications.

In benchmarking by Artificial Analysis, Groq 3 LPX delivered 3,400 output tokens per second running the Gemma 4 31B model with a 100,000-token context. NVIDIA says this represents the fastest performance recorded for that model and provides up to 4× faster responsiveness for agentic and other latency-sensitive workloads compared with the nearest alternative platform.

The architecture combines NVIDIA Rubin GPUs, which provide high-bandwidth memory capacity, with Groq language-processing units built around high-speed SRAM. Each LPX rack incorporates 256 LPU accelerators, with 128GB of SRAM and aggregate SRAM bandwidth of up to 40PB/s per rack, enabling deterministic low-latency processing.

Nebius is the first AI cloud provider planning to deploy Groq 3 LPX in production through its Nebius Token Factory inference platform, while Groq is also expected to be among the platform’s early adopters.

The move highlights a broader change in AI infrastructure design: rather than relying on a single accelerator architecture for every workload, next-generation AI systems are increasingly combining specialized processors for training, context processing and high-speed token generation.

Source: NVIDIA
Photo credit: NVIDIA