MLCommons has extended its inference benchmark suite to include a power-measurement rating. MLPerf 1.0 allows submitters to report the system power while running each test, enabling power-efficiency comparisons. For simplicity, AC power is measured at the wall, so it encompasses the processor, accelerator, memory, and other semiconductor components as well as the power converter and cooling system, if any. This procedure offers some visibility into the actual power consumed by leading accelerators.

Reporting the measured power is optional, so few vendors did so; the new release includes only six results for the data-center power tests and seven for the edge tests in the closed division. These results validate Qualcomm’s claims that its Cloud AI accelerators are more power efficient than Nvidia’s best data-center and edge chips, but only on the two benchmarks Qualcomm submitted.

Of the 117 closed results, which must strictly adhere to MLPerf rules, most feature Nvidia accelerators or edge processors (e.g., Xavier); some employ the Qualcomm accelerator or rely entirely on Intel or Arm CPUs. No high-profile startup submitted results. Other than Nvidia and Intel, vendors submitted just one or two tests that favored their product, rather than providing results for the entire suite.

Thus, MLPerf remains primarily a tool for detailed comparisons among various Nvidia accelerators and a method of certifying ResNet-50 results. On that neural network, Nvidia’s new 80GB version of the Ampere A100 SXM module leads the pack at nearly 40,000 inferences per second (IPS). The A100 PCIe card, which has 40GB of HBM and is limited to 300W, achieves a 17% lower score. Impressively, Qualcomm’s PCIe card, which is rated at only 75W, delivers more than half of the A100’s ResNet performance.