» Current | 2022 | 2021 | 2020

Linley Newsletter

MLPerf 1.0 Adds Power Measurements

June 8, 2021

Author: Linley Gwennap

MLCommons has extended its inference benchmark suite to include a power-measurement rating. MLPerf 1.0 allows submitters to report the system power while running each test, enabling power-efficiency comparisons. For simplicity, AC power is measured at the wall, so it encompasses the processor, accelerator, memory, and other semiconductor components as well as the power converter and cooling system, if any. This procedure offers some visibility into the actual power consumed by leading accelerators.

Reporting the measured power is optional, so few vendors did so; the new release includes only six results for the data-center power tests and seven for the edge tests in the closed division. These results validate Qualcomm’s claims that its Cloud AI accelerators are more power efficient than Nvidia’s best data-center and edge chips, but only on the two benchmarks Qualcomm submitted.

Of the 117 closed results, which must strictly adhere to MLPerf rules, most feature Nvidia accelerators or edge processors (e.g., Xavier); some employ the Qualcomm accelerator or rely entirely on Intel or Arm CPUs. No high-profile startup submitted results. Other than Nvidia and Intel, vendors submitted just one or two tests that favored their product, rather than providing results for the entire suite.

Thus, MLPerf remains primarily a tool for detailed comparisons among various Nvidia accelerators and a method of certifying ResNet-50 results. On that neural network, Nvidia’s new 80GB version of the Ampere A100 SXM module leads the pack at nearly 40,000 inferences per second (IPS). The A100 PCIe card, which has 40GB of HBM and is limited to 300W, achieves a 17% lower score. Impressively, Qualcomm’s PCIe card, which is rated at only 75W, delivers more than half of the A100’s ResNet performance.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products


Linley Spring Processor Conference 2022
Registration now open: April 20-21, 2022
Hyatt Regency Hotel, Santa Clara, CA
Linley Fall Processor Conference 2021
Held October 20-21, 2021
Proceedings available
Linley Spring Processor Conference 2021
April 19 - 23, 2021
Proceedings Available
More Events »