“We Made a Chip and It Is Fast”: Inside OpenAI’s New ‘Jalapeño’ Silicon

“We made a chip, and it is fast.”

With that simple, understated post on X, OpenAI CEO Sam Altman officially marked a massive turning point for the AI giant. The company behind ChatGPT is currently building the very hardware their AI models run on.

Developed in partnership with Broadcom and playfully dubbed “Jalapeño,” OpenAI’s first custom artificial intelligence processor has arrived. The chip is already posting benchmark results that are turning heads across the industry.

Here is a breakdown of what Jalapeño is, how it performs, and why it matters for the future of AI.

To understand Jalapeño, you have to understand the two main phases of an AI model’s lifecycle: training and inference.

Training is the heavy, glamorous work of teaching an AI model from scratch, a process that requires tens of thousands of GPUs running for months. Jalapeño is not built for this.

Instead, Jalapeño is purpose-built for inference, the unglamorous but incredibly expensive job of actually answering your prompts, writing code, and generating responses once the model already exists. Every time a user interacts with ChatGPT or Codex, an inference chip is doing the work. By building a processor that does exactly one thing at massive scale, OpenAI aims to shift its unit economics and power consumption fundamentally.

According to recent benchmarks verified by SemiAnalysis, Jalapeño is a major technological leap. When tested on the InferenceX benchmark against current-generation Nvidia systems (like the GB200 and GB300), OpenAI’s new silicon delivered impressive numbers:

OpenAI’s Vice President of Hardware, Richard Ho, noted that standard AI chips usually force a trade-off between throughput (handling lots of requests at once) and latency (speed of a single response). Jalapeño’s custom architecture circumvents this by minimising data movement, keeping the model’s key-value cache locally on the chip.

Nvidia has enjoyed a near-monopoly on the AI boom, supplying the vast majority of the world’s highly coveted GPUs. OpenAI has historically been one of Nvidia’s biggest and most dependent customers.

Jalapeño represents a major shift toward OpenAI becoming a “full-stack” AI company. While OpenAI has confirmed they will continue relying on Nvidia chips for training and as a compute partner, moving inference workloads to in-house silicon will drastically reduce their reliance on a single supplier. They are joining the ranks of Google (TPUs), Amazon (Trainium/Inferentia), and Microsoft (Maia), all of which have invested heavily in proprietary silicon.

As models get smarter, the physical infrastructure running them must keep pace. With Jalapeño, OpenAI is proving they intend to control their destiny down to the very silicon their AI calls home.