OpenAI has offered a clearer look at Jalapeño, its new chip system for AI inference, and the message is direct: the company is trying to make large-scale AI responses faster and more power-efficient.
At the Hot Chips conference on Tuesday, OpenAI shared the first batch of benchmark results for Jalapeño. Tested on SemiAnalysis’ InferenceX benchmark, the system registered more tokens per user and more throughput per kilowatt than the currently available state-of-the-art inference processors.
What The Benchmarks Show
The central claim is that Jalapeño can handle more AI work while using power more efficiently. Inference is the phase where an AI system generates responses for users, so speed, latency, and energy use matter directly to how many people can be served and how quickly they receive results.
Richard Ho, OpenAI’s head of hardware, described the results as a major step forward in a press call. “The bottom line is that the results show a very, very significant performance advance over state of the art,” he said.
Ho also tied the benchmark performance to practical service demands. “Jalapeño can serve more AI work per unit of power, while also returning responses more quickly. It’s very efficient to serve a lot of customers, but it can also be very low latency.”
Those two goals are closely linked. A system that produces more throughput per kilowatt can support more inference work for the same power budget. A system that returns responses more quickly can improve the experience for users who expect AI products to feel immediate.
The Comparison Comes With Timing Limits
The benchmark comparison is notable because it is against an Nvidia Blackwell system. That gives the results a meaningful reference point against currently available state-of-the-art inference processors.
But the timeline also matters. Ho estimated that Jalapeño would deploy at the end of 2026 “in very small volumes,” with more significant deployment coming in 2027. By then, the competitive field may have moved forward.
That means the results should be read as an early signal rather than a final market verdict. Jalapeño appears designed around OpenAI’s inference needs, but its real-world significance will depend on how it performs when deployed and what competing systems look like at that point.
Why OpenAI Is Building Its Own Inference Platform
Jalapeño was first announced last October. It was developed by OpenAI in close collaboration with Broadcom, and OpenAI’s own models assisted in the development process.
The chip is not being framed as a one-off component. OpenAI plans to make Jalapeño a multigenerational platform. That approach is meant to let AI products, models, chips, and memory be developed together rather than treated as separate layers.
That full-stack approach is important because inference performance is not only about raw compute. Bottlenecks can appear when data has to move across parts of a system or when communication between components slows down the response process.
OpenAI says Jalapeño is aimed at specific phases of inference that often create friction. In particular, the system is designed to reduce delays during the prefill and communication phases of processing.
How Jalapeño Targets Inference Bottlenecks
OpenAI’s explanation centers on keeping data closer to where it is needed. In a blog post presenting the results, the company said, “We designed Jalapeño to minimize data movement and communication delays.”
The company also described how the system handles model state during response generation. “This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase.”
In plain terms, OpenAI is trying to reduce the amount of unnecessary movement inside the system. If the data needed for a response can stay local, the chip can spend less time waiting on communication and more time serving inference work.
That matters most at scale. AI services must handle many users at once, and small delays inside the system can become larger operational constraints when multiplied across heavy usage.
What To Watch Next
The most important next step is deployment. OpenAI has said very small volumes are expected at the end of 2026, with broader use coming in 2027. Until then, Jalapeño’s benchmark results show the direction of OpenAI’s hardware strategy more clearly than they prove its long-term position in the market.
The strategy is clear enough: build an inference system around OpenAI’s own models and product needs, tune it for low latency and power efficiency, and treat chips, memory, networking, and models as connected parts of the same platform.
If that approach works in deployment, Jalapeño could become a central part of how OpenAI serves AI products at scale. For now, it is an early benchmark-backed look at a chip built for one of the biggest constraints in modern AI: delivering more responses, faster, without power use becoming the limiting factor.