» Current | 2021 | 2020 | 2019 | Subscribe

Linley Newsletter

Deep Vision Optimizes AI Data Flow

March 9, 2021

Author: Markus Levy

Deep Vision’s first AI chip, the ARA-1 inference coprocessor, introduces the company’s patented polymorphic data-flow architecture. Its main goal is to minimize on-chip data movement by combining a software-managed scheduler with programmable execution cores. The design assumes each layer of a neural-network model has a different “best” computation and data-movement approach; in other words, no single configuration is efficient for all. Therefore, instead of implementing a predefined acceleration scheme in hardware, the silicon makes available to the software a set of flexible computation cores and data-management engines. The software then employs these units to follow the optimal execution plan for each neural-network subgraph.

The combined architecture and compiler allow the ARA-1 to run ResNet-50 models at 120 inferences per second (with batch size =1) and 2W, all for less than $20. Since typical camera rates are 30fps to 60fps, this performance leaves plenty of headroom to implement additional models or more-complex models. In addition to ResNet-50, the compiler can digest most vision-based networks developed from Keras, TensorFlow, PyTorch, Caffe, or MXNet frameworks. The software-centric approach is central to Deep Vision’s goal of broad support for neural-network architectures and operators.

The company received first ARA-1 silicon in 2Q19 and expects volume production to begin this month. In addition to standalone chips, it will sell USB modules, M.2 modules, and PCIe cards. The chip is built in TSMC 28nm HPC technology with a standard-cell library, and it consumes 1.7–2.3W (typical), depending on the network (and whether it is compiled for high performance or low energy). The chip implements eight compute engines that we estimate to deliver nearly 4 TOPS. An integrated 4MB SRAM feeds the execution cores: it occupies about 10% of the die area and is crucial to reducing power.

Subscribers can view the full article in the Microprocessor Report.

Subscribe to the Microprocessor Report and always get the full story!

Purchase the full article

Free Newsletter

Linley Newsletter
Analysis of new developments in microprocessors and other semiconductor products
Subscribe to our Newsletter »

Events

Linley Spring Processor Conference 2021
April 19 - 23, 2021
Virtual Event
More Events »