Kicking off one other busy Spring GPU Technology Conference for NVIDIA, this morning the graphics and accelerator designer is saying that they will as soon as once more design their very own Arm-based CPU/SoC. Dubbed Grace – after Grace Hopper, the pc programming pioneer and US Navy rear admiral – the CPU is NVIDIA’s newest stab at extra absolutely vertically integrating their {hardware} stack by having the ability to supply a high-performance CPU alongside their common GPU wares. According to NVIDIA, the chip is being designed particularly for large-scale neural community workloads, and is anticipated to develop into out there in NVIDIA merchandise in 2023.
With two years to go till the chip is prepared, NVIDIA is taking part in issues comparatively coy at the moment. The firm is providing solely restricted particulars for the chip – it will likely be based mostly on a future iteration of Arm’s Neoverse cores, for instance – as immediately’s announcement is a little more targeted on NVIDIA’s future workflow mannequin than it’s speeds and feeds. If nothing else, the corporate is making it clear early on that, a minimum of for now, Grace is an inner product for NVIDIA, to be provided as a part of their bigger server choices. The firm isn’t straight gunning for the Intel Xeon or AMD EPYC server market, however as an alternative they’re constructing their very own chip to enrich their GPU choices, making a specialised chip that may straight hook up with their GPUs and assist deal with monumental, trillion parameter AI fashions.
| NVIDIA SoC Specification Comparison | |||||
| Grace | Xavier | Parker (Tegra X2) |
|||
| CPU Cores | ? | 8 | 2 | ||
| CPU Architecture | Next-Gen Arm Neoverse (Arm v9?) |
Carmel (Custom Arm v8.2) |
Denver 2 (Custom Arm v8) |
||
| Memory Bandwidth | >500GB/sec LPDDR5X (ECC) |
137GB/sec LPDDR4X |
60GB/sec LPDDR4 |
||
| GPU-to-CPU Interface | >900GB/sec NVLink 4 |
PCIe 3 | PCIe 3 | ||
| CPU-to-CPU Interface | >600GB/sec NVLink 4 |
N/A | N/A | ||
| Manufacturing Process | ? | TSMC 12nm | TSMC 16nm | ||
| Release Year | 2023 | 2018 | 2016 | ||
More broadly talking, Grace is designed to fill the CPU-sized gap in NVIDIA’s AI server choices. The firm’s GPUs are extremely well-suited for sure lessons of deep studying workloads, however not all workloads are purely GPU-bound, if solely as a result of a CPU is required to maintain the GPUs fed. NVIDIA’s present server choices, in flip, usually depend on AMD’s EPYC processors, that are very quick for basic compute functions, however lack the type of high-speed I/O and deep studying optimizations that NVIDIA is on the lookout for. In explicit, NVIDIA is at the moment bottlenecked by way of PCI Express for CPU-GPU connectivity; their GPUs can speak shortly amongst themselves by way of NVLink, however not again to the host CPU or system RAM.
The answer to the issue, as was the case even earlier than Grace, is to make use of NVLink for CPU-GPU communications. Previously NVIDIA has labored with the OpenPOWER basis to get NVLink into POWER9 for precisely this motive, nonetheless that relationship is seemingly on its means out, each as POWER’s recognition wanes and POWER10 is skipping NVLink. Instead, NVIDIA goes their very own means by constructing an Arm server CPU with the mandatory NVLink performance.
The finish end result, in accordance with NVIDIA, shall be a high-performance and high-bandwidth CPU that’s designed to work in tandem with a future era of NVIDIA server GPUs. With NVIDIA speaking about pairing every NVIDIA GPU with a Grace CPU on a single board – much like immediately’s mezzanine playing cards – not solely does CPU efficiency and system reminiscence scale up with the variety of GPUs, however in a roundabout means, Grace will function a co-processor of kinds to NVIDIA’s GPUs. This, if nothing else, is a really NVIDIA answer to the issue, not solely enhancing their efficiency, however giving them a counter ought to…
