» Current | 2022 | 2021 | 2020

Linley Newsletter

Nvidia Orin Appears in MLPerf

April 19, 2022

Author: Linley Gwennap

The newest set of MLPerf Inference results showcase the same old vendors; almost all the data-center and edge accelerators came from Nvidia and Qualcomm. The one notable newcomer was Nvidia’s Orin processor, which targets automotive and edge equipment.

Officially in the “preview” category until production shipments in 2H22, Orin achieved 6,139 images per second (IPS) on ResNet-50 v1.5 Offline, less than a low-end Ampere A10 card but using much less power. In fact, Orin scores twice Ampere’s IPS per watt on ResNet, though its lead shrinks on larger benchmarks that stress the edge processor’s diminutive cache.

Qualcomm’s Cloud AI 100 DLA trounces Orin on ResNet at 23,808 IPS. At that speed, however, it uses 75W, and the comparison is unfair since the Nvidia chip lacks host CPUs. To even things up, Qualcomm paired a Snapdragon host processor with a 20W Cloud AI chip for the MLPerf power comparison; this combination nearly triples Orin’s system-level ResNet IPS per watt while also topping Orin’s efficiency on the other three benchmarks Qualcomm submitted.

By packing 18 PCIe cards into a single 2U server, Qualcomm generated greater ResNet-50 performance than a 6U DGX-A100 system with eight Ampere cards. This comparison shows the Cloud AI 100 can deliver greater performance density, at least for image-processing models.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products


Linley Spring Processor Conference 2022
Registration now open: April 20-21, 2022
Hyatt Regency Hotel, Santa Clara, CA
Linley Fall Processor Conference 2021
Held October 20-21, 2021
Proceedings available
Linley Spring Processor Conference 2021
April 19 - 23, 2021
Proceedings Available
More Events »