Nvidia has moved the Groq 3 LPX into full production, positioning the accelerator as a high-speed engine for AI inference. The company says the system can generate tokens fast enough to make agentic AI workflows feel far more responsive.
The headline number is eye-catching: 3,400 tokens per second in an independent benchmark. But the practical meaning of that result is more complicated than a simple race between two chips.
What Nvidia announced
At Hot Chips 2026, Nvidia said the Groq 3 LPX has entered full production. Nvidia describes the chip as an "interactive AI inference accelerator" and says it extends the Vera Rubin platform.
The accelerator is aimed at fast token generation for agentic AI systems. Nvidia says it will go live later this year.
The move follows Nvidia’s late December deal to pay about $20 billion for the Groq license and bring on founder Jonathan Ross and president Sunny Madra. Groq’s processors are tuned for inference rather than AI training, which makes the technology relevant to systems that need to produce answers quickly after a model has already been built.
Why token speed matters for AI agents
Agentic applications can use huge amounts of tokens across hundreds to thousands of inference steps. Each step may involve reasoning, checking context, calling tools, writing code, testing changes or verifying a result.
That makes output speed important. If a system can generate more tokens inside the same acceptable wait time, the agent can perform more work before the user has to wait too long.
Nvidia’s argument is that faster token generation gives AI agents room to iterate more often. In coding tasks, the company says this can reduce work to "minutes instead of hours." The claim is not only about raw speed; it is about fitting more reasoning and tool use into the same user experience.
The benchmark result
A benchmark from Artificial Analysis tested the Groq 3 LPX on the open model Gemma 4 31B with a 100,000-token context window. In that setup, the LPX rack reached 3,400 tokens per second across 50 back-to-back requests.
Performance held steady between 10,000 and 100,000 tokens of input length. The source describes this as the highest figure ever recorded for this model.
Nvidia says the result makes the accelerator four times faster than the next best option, the Cerebras chip, at 882 tokens per second. On the surface, that looks like a clear performance gap. The harder question is whether the systems being compared are equivalent in practice.
Where the comparison gets complicated
The Groq 3 LPX uses an SRAM-heavy dataflow architecture. According to The Register, each LPU has just 500 MB of memory, which is 576 times less than a Rubin GPU with 288 GB.
Because of that memory limit, models are split across several accelerators over Ethernet. A single rack can hold up to 256 LPUs. In the setup described, GPUs handle the compute-heavy prefill phase, while LPUs handle the bandwidth-heavy decode phase.
That division matters because the benchmark is not simply one accelerator doing the whole job in isolation. It is a system-level result from a rack design that combines different components for different stages of inference.
The Register also points out that Gemma 4 31B is a favorable case for this architecture because it is a dense model that fits entirely in one rack. How the same approach scales with larger mixture-of-experts models remains unresolved in the source. DeepSeek V3, for example, would need 1,342 accelerators, or a little over five racks.
The Cerebras comparison also leaves out chip counts. Cerebras requires one or two accelerators for the model, while Nvidia needs at least 64. The comparison also does not include Cerebras’ newest CS-4 generation.
What happens next
The result gives Nvidia a strong performance story for the Groq 3 LPX, especially around high-speed inference for AI agents. But the benchmark should be read as a rack-level demonstration under a setup that appears well suited to the chosen model.
For customers, the important questions are likely to be practical ones:
- How many accelerators are needed for the target model?
- How does performance change with larger mixture-of-experts models?
- How much infrastructure is required to reach the advertised speed?
- How does the comparison change when newer Cerebras systems are considered?
Nebius plans to be the first cloud provider to offer the chip through its Token Factory. Groq itself is also among the early users.
For now, Nvidia’s Groq 3 LPX appears to set a notable benchmark for token generation. The caveat is that the headline speed number does not tell the whole story without memory, architecture, rack scale and chip count in view.