Nvidia used GTC 2024 to place Blackwell at the center of its generative AI roadmap. The company is presenting the new GPU platform as hardware built for large language models that can reach up to several trillion parameters, with major gains in training, inference, efficiency, and system scale.
Why Blackwell matters for generative AI
According to Nvidia CEO Jensen Huang, Blackwell is meant to become a driving force in a new industrial revolution. That ambition rests on a simple premise: bigger and more capable generative AI systems need hardware that can move data quickly, connect many GPUs efficiently, and process large models with lower precision formats where appropriate.
Nvidia says Blackwell includes the world's most powerful chip with 208 billion transistors. The design combines two dies made using TSMC's 4NP process. Those dies are linked at 10TB/second, allowing them to work together as a single CUDA GPU.
The platform is not only about raw compute. Nvidia is also emphasizing the surrounding architecture, including a second-generation Transformer Engine, enhanced NVLink communication, a RAS Engine for AI predictive maintenance, and a dedicated decompression engine designed to accelerate database queries.
The headline performance claims
For AI computing, the Blackwell GPU is rated at 10 petaFLOPS in FP8 and 20 petaFLOPS in FP4. It also carries 192 gigabytes of HBM3e memory, giving it a large memory pool for demanding AI workloads.
The new Transformer Engine supports AI applications with FP4 accuracy. Nvidia says that when it is used with “Micro Tensor Scaling”, the result can be twice the computing power, twice the model size, and twice the bandwidth.
Compared with the H100 GPU, Nvidia says Blackwell can deliver:
- four times the training performance
- up to 25 times the power efficiency
- up to 30 times the inference performance
The largest inference gain applies to mixture-of-expert models such as GPT-4. For classical large transformation models such as GPT-3, the increase is 7x. That difference matters because Nvidia is tying the biggest jump to a model type that depends heavily on efficient communication between GPUs.
The source of that improvement is the new NVLink and NVLink Switch 7.2. Nvidia says these technologies improve communication between GPUs, which has previously been a bottleneck for mixture-of-expert models. As MoE models become more important, with Google's Gemini also based on this principle, faster interconnects become a central part of the AI hardware story.
Cloud providers are in the first wave
Nvidia expects Blackwell to be used by nearly all major cloud providers and server manufacturers. The first companies expected to deploy the platform include Amazon Web Services, Google, Meta, Microsoft, and OpenAI.
That early customer list shows where Nvidia sees the platform fitting first: large infrastructure environments that need to train, tune, and serve generative AI models at scale. The company is also positioning Blackwell against inference-focused chips that are trying to take market share from Nvidia's existing GPU business.
Inference is the stage where an AI model generates outputs after training. For large language models, that can become a major performance and efficiency challenge, especially when many users are sending prompts at the same time. Nvidia's up to 30 times inference claim for GPT-4-level mixture-of-expert models is therefore one of the most important parts of the Blackwell announcement.
DGX SuperPOD moves to the Blackwell generation
Blackwell is also arriving inside a new DGX SuperPOD. Nvidia describes the system as a liquid-cooled, rack-scale architecture that delivers 11.5 ExaFLOPS of FP4-precision AI supercomputing performance and 240 terabytes of fast memory.
The system can scale to tens of thousands of chips by adding racks. At its center is the GB200 NVL72, which links 36 Nvidia GB200 supercomputer chips. Each includes 36 Grace CPUs and 72 Blackwell GPUs, connected through Nvidia's fifth-generation NVLink to form a supercomputer.
According to Nvidia, the GB200 Super Chips can deliver up to 30 times the performance of the same number of Nvidia H100 Tensor Core GPUs for inference workloads with large language models.
Huang described one DGX GB200 NVL72, thanks to the new NVLink chip, as basically “on giant GPU”. The system delivers 720 PetaFLOPS for training FP8 and 1,44 ExaFLOPS for inference in FP4.
DGX B200 rounds out the platform
Nvidia also introduced the DGX B200 system for AI model training, tuning, and inference. It is the sixth generation of the company's air-cooled DGX design and connects eight B200 Tensor Core GPUs to CPUs.
Both the DGX SuperPOD generation and DGX B200 are expected to be available later this year. Together, they show how Nvidia is packaging Blackwell not only as a chip, but as a full platform for organizations building and running large generative AI systems.