» Current | 2022 | 2021 | 2020

Linley Newsletter

Nvidia Hopper Leaps Ahead

April 12, 2022

Author: Linley Gwennap

For two years, other AI-accelerator vendors have tried and failed to surpass Nvidia’s Ampere A100. Now, the bar is suddenly much higher, as the market leader introduced its newest AI processor, the Hopper H100. In its maximum configuration, the chip triples raw performance over the A100 while pushing power consumption to a scorching 700W. New features such as 8-bit floating-point (FP8) support and improved scalability promise even greater gains for training large networks.

The Hopper architecture doubles the number of MAC units per core, boosting performance at modest power gain. Using 4nm technology, the GPU packs more cores than the A100 while raising the clock speed as well. The new FP8 format doubles throughput compared with standard FP16 training.

The company offers the H100 in a custom SXM5 module or a standard PCIe card; the latter delivers about one-third less performance at 350W. Nvidia has also developed a new DGX-H100 system that combines eight Hopper chips. It employs next-generation NVLink technology to create high-bandwidth connections between systems, greatly increasing performance in clusters of up to 32 DGX systems. The company expects production shipments of H100-based modules, cards, and systems by September.

At the recent GTC, Nvidia also gave a brief update on its Grace processor, which is due to ship in 1H23. Grace will contain 72 Arm Neoverse CPUs and 198MB of on-chip cache while connecting to eight LPDDR5X channels. The processor will typically sit side by side with a Hopper GPU, forming what Nvidia calls the “Grace Hopper superchip” (after the celebrated admiral and computer scientist). In this configuration, the processor can feed data from 512GB of DRAM directly to the GPU through its coherent NVLink connections, accelerating performance on enormous models such as GPT.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products

Events

Linley Spring Processor Conference 2022
Registration now open: April 20-21, 2022
Hyatt Regency Hotel, Santa Clara, CA
Linley Fall Processor Conference 2021
Held October 20-21, 2021
Proceedings available
Linley Spring Processor Conference 2021
April 19 - 23, 2021
Proceedings Available
More Events »