» Current | 2020 | 2019 | 2018 | Subscribe

Linley Newsletter

Untether Delivers At-Memory AI

November 3, 2020

Author: Linley Gwennap

Using what it calls an at-memory architecture, Untether AI has created a highly power-efficient accelerator that can achieve a stunning two petaop/s in a PCIe card. The at-memory design interleaves more than 250,000 tiny processing elements (PEs) inside a standard SRAM array. Putting the processing next to the memory enables massive data flow into the PEs. It also greatly reduces the power required to move data to the compute units; this movement consumes more power than the computation itself in traditional architectures. The startup is already sampling its TsunAImi card, which accelerates AI inference, and expects to ship production-qualified units in 2Q21.

Untether emerged from stealth and introduced its new architecture at the recent Linley Fall Processor Conference. The RunAI200 chip features 511 cores that contain a total of 192MB of primary SRAM, obviating the need for external DRAM. Operating at 960MHz, the cores generate peak performance of 502 trillion operations per second (TOPS) for the 8-bit integer (INT8) operations that commonly serve in neural-network inferencing. At a relatively low 100W TDP, RunAI200 offers more than 3x better TOPS per watt than Nvidia’s new Ampere A100 GPU. The new chip also supports an “eco mode” that delivers 377 TOPS at 47W.

To boost performance, the TsunAImi card includes four of these chips. At 2,000 TOPS, it outperforms all other high-end AI accelerators by more than 2x. To achieve this rating, the card has a 400W TDP, similar to the A100 card. Boasting nearly 800MB of on-chip memory, it can hold large models without using any DRAM. Untether expects the four-chip design to achieve 80,000 images per second (IPS) on ResNet-50, which would be a record for a single accelerator card. Alibaba’s HanGuang 800 card is rated at 78,563 IPS.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Purchase the full article

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products
Subscribe to our Newsletter »

Events

Linley Spring Processor Conference 2021
Coming April 21-22, 2021
Hyatt Regency Hotel, Santa Clara, CA
More Events »