We’ve recognized about Intel’s Cooper Lake platform for plenty of quarters. What was initially deliberate, so far as we perceive, as a customized silicon variant of Cascade Lake for its high-profile prospects, it was subsequently productized and aimed to be inserted right into a delay in Intel’s roadmap brought on by the event of 10nm for Xeon. Set to be a full vary replace to the product stack, within the final quarter, Intel declared that its Cooper Lake platform would find yourself solely within the arms of its precedence prospects, solely as a quad-socket or increased platform. Today, Intel launches Cooper Lake, and confirms that Ice Lake is ready to return out later this 12 months, aimed on the 1P/2P markets.
Count Your Coopers: BFloat16 Support
Cooper Lake Xeon Scalable is formally designated as Intel’s 3rd Generation of Xeon Scalable for high-socket rely servers. Ice Lake Xeon Scalable, when it launches later this 12 months, may even be known as 3rd Generation of Xeon Scalable, apart from low core rely servers.
For Cooper Lake, Intel has made three key additions to the platform. First is the addition of AVX512-based BF16 directions, permitting customers to reap the benefits of the BF16 quantity format. Plenty of key AI workloads, usually achieved in FP32 or FP16, can now be carried out in BF16 to get nearly the identical throughput as FP16 for nearly the identical vary of FP32. Facebook made an enormous deal about BF16 in its presentation final 12 months at Hot Chips, the place it varieties a crucial a part of its Zion platform. At the time the presentation was made, there was no CPU available on the market that supported BF16, which led to this amusing change on the convention:
BF16 (bfloat16) is a method of encoding a quantity in binary that makes an attempt to reap the benefits of the vary of a 32-bit quantity, however in a 16-bit format such that double the compute could be packed into the identical variety of bits. The easy desk appears to be like a bit like this:
| Data Type Representations | ||||||
| Type | Bits | Exponent | Fraction | Precision | Range | Speed |
| float32 | 32 | 8 | 23 | High | High | Slow |
| float16 | 16 | 5 | 10 | Low | Low | 2x Fast |
| bfloat16 | 16 | 8 | 7 | Lower | High | 2x Fast |
By utilizing BF16 numbers fairly than FP32 numbers, it might additionally imply that reminiscence bandwidth necessities in addition to system-to-system community necessities may very well be halved. On the size of a Facebook, or an Amazon, or a Tencent, this is able to attraction to them. At the time of the presentation at Hot Chips final 12 months, Facebook confirmed that it already had silicon engaged on its datasets.
Doubling Socket-to-Socket Interconnect Bandwidth
The second improve that Intel has made to Cooper Lake over Cascade Lake is in socket-to-socket interconnect. Traditionally Intel’s Xeon processors have relied on a type of QPI/UPI (Ultra Path Interconnect) as a way to join a number of CPUs collectively to behave as one system. In Cascade Lake Xeon Scalable, the highest finish processors every had three UPI hyperlinks working at 10.four GT/s. For Cooper Lake, we’ve got six UPI hyperlinks additionally working at 10.four GT/s, nevertheless these hyperlinks nonetheless solely have three controllers behind them such that every CPU can solely join to a few different CPUs, however the bandwidth could be doubled.
This signifies that in Cooper Lake, every CPU-to-CPU connection entails two UPI hyperlinks, every working at 10.four GT/s, for a complete of 20.Eight GT/s. Because the variety of hyperlinks is doubled, fairly than an evolution of the usual, there are not any energy effectivity enhancements past something Intel has achieved to the manufacturing course of. Note that double the bandwidth between sockets continues to be a superb factor, even when latency and energy per bit continues to be the identical.
Intel nonetheless makes use of the double pinwheel topology for its eight socket…
