Blackwell (microarchitecture)

Blackwell is a graphics processing unit (GPU) microarchitecture developed by Nvidia as the successor to the Hopper and Ada Lovelace microarchitectures.

Named after statistician and mathematician David Blackwell, the name of the Blackwell architecture was leaked in 2022 with the B40 and B100 accelerators being confirmed in October 2023 with an official Nvidia roadmap shown during an investors presentation.^[1] and was officially announced at Nvidia's GTC 2024 keynote on March 18, 2024.^[2]

History[edit]

In March 2022, Nvidia announced Hopper architecture for datacenter for AI accelerators. Demand for Hopper products was high throughout 2023's AI hype.^[3] The lead time from order to delivery of H100-based servers was between 36 and 52 weeks due to shortages and high demand.^[4] Nvidia reportedly sold 500,000 Hopper-based H100 accelerators in Q3 2023 alone.^[4] Nvidia's AI dominance with Hopper products led to the company increasing its market capitalization to over $2 trillion, behind only Microsoft and Apple.^[5]

The Blackwell architecture is named after American mathematician David Blackwell who was known for his contributions to the mathematical fields of game theory, probability theory, information theory, and statistics. These areas have influenced or are implemented in transformer-based generative AI model designs or their training algorithms. Blackwell was the first African American scholar to be inducted into the National Academy of Sciences.^[6]

In Nvidia's October 2023 Investor Presentation, its datacenter roadmap was updated to include reference to its B100 and B40 accelerators and the Blackwell architecture.^[7][8] Previously, the successor to Hopper was simply named on roadmaps as "Hopper-Next". Nvidia's updated roadmap emphasized the move from a two-year release cadence for datacenter products to yearly releases targeted for x86 and ARM systems.

At the Graphics Technology Conference (GTC) on March 18, 2024, Nvidia officially announced the Blackwell architecture with focus placed on its B100 and B200 datacenter accelerators. Nvidia CEO Jensen Huang said that with Blackwell, "we created a processor for the generative AI era" and emphasized the overall Blackwell platform combining Blackwell accelerators with Nvidia's ARM-based Grace CPU.^[9][10] Nvidia touted endorsements of Blackwell from the CEOs of Google, Meta, Microsoft, OpenAI and Oracle.^[10] The keynote did not mention gaming.

Architecture[edit]

Blackwell is an architecture designed for both datacenter compute applications and for gaming and workstation applications with dedicated dies for each purpose. The GB100 die is for Blackwell datacenter products while GB200 series dies will be used for GeForce RTX 50 series graphics cards.

Process node[edit]

Blackwell is fabricated on the custom 4NP node from TSMC. 4NP is an enhancement of the 4N node used for the Hopper and Ada Lovelace architectures with an increase in transistor density. With the enhanced 4NP node, the GB100 die contains 104 billion transistors, a 30% increase over the 80 billion transistors in the previous generation Hopper GH100 die.^[11] As Blackwell cannot reap the benefits that come with a major process node advancement, it must achieve power efficiency and performance gains through underlying architectural changes.^[12]

The GB100 die is at the reticle limit of semiconductor fabrication.^[13] The reticle limit in semiconductor fabrication is the physical size limit that lithography machines can etch a silicon die. Previously, Nvidia had nearly hit TSMC's reticle limit with GH100's 814mm² die. In order to not be constrained by die size, Nvidia's B100 accelerator utilizes two GB100 dies in a single package, connected with a 10 TB/s link that Nvidia calls the NV-High Bandwidth Interface (NV-HBI). NV-HBI is based on the NVLink 5.0 protocol. Nvidia CEO Jensen Huang claimed in an interview with CNBC that Nvidia had spent around $10 billion in research and development for Blackwell's NV-HBI die interconnect. Veteran semiconductor engineer Jim Keller, who had worked on AMD's K7, K12 and Zen architectures, criticized this figure and claimed that the same outcome could be achieved for $1 billion through using Ultra Ethernet rather than the proprietry NVLink system.^[14] The two connected GB100 dies are able to act like a large monolithic piece of silicon with full cache coherency between both dies.^[15] The dual die package totals 208 billion transistors.^[13] Those two GB100 dies are placed on top on a silicon interposer produced using TSMC's CoWoS-L 2.5D packaging technique.^[16]

Streaming Multiprocessor[edit]

CUDA Cores[edit]

CUDA Compute Capability 10.0 is added with Blackwell.

Tensor Cores[edit]

The Blackwell architecture introduces fifth generation Tensor Cores for AI compute and performing floating-point calculations. In the datacenter, Blackwell adds support for FP4 and FP6 data types with eighth precision floating point processing.^[17] The previous Hopper architecture introduced the Transformer Engine to divide FP32 data into FP8 to increase peak compute throughput. Blackwell's second generation Transformer Engine allows FP32 to be divided further, allow a doubling of FP8 compute performance. Using 4-bit data allows greater efficiency and throughput for model inference during generative AI training.^[12] Nvidia claims 20 petaflops of FP4 compute with the dual GB100 die B100 accelerator.^[18]

See also[edit]

• List of eponyms of Nvidia GPU microarchitectures

References[edit]