The development of AI has seen a resurgence in enterprise capital funding for silicon start-ups. Designing AI silicon for machine studying, each for coaching and inference, has grow to be sizzling property in Silicon Valley, particularly as machine studying compute and reminiscence necessities are coalesced into tangible targets for this silicon to go after. A variety of these corporations are already transport excessive efficiency processors to prospects, and are in search of additional funding to assist help prospects, develop the client base, and develop subsequent technology merchandise till profitability occurs, or the corporate is acquired. The two newest funding rounds for AI silicon have been introduced on this previous week.
Groq (Series C, $300m, Tensor Streaming Processor Q100)
When Groq’s first product got here onto the scene, detailed by the Microprocessor Report again in January 2020, it was described as the primary PetaOP processor that eschewed conventional many-core designs and as an alternative applied a single VLIW-like core with a whole lot of useful items. In this methodology, the info is topic to instruction circulation, slightly than directions being reliant on information circulation, saving time on synchronicity and decode overhead that many-core processors require.
The finish result’s a product that implements 400,000 multiply-accumulate items, however the important thing advertising and marketing metric is the deterministic efficiency. Using this single core methodology, Groq’s Q100 TSP will take the identical time to inference workload with none quality-of-service necessities. In talking with CEO Jonathan Ross, Groq’s TSP allows workloads that have been beforehand unusable because of lengthy tail high quality of service efficiency degradation (i.e. worst case outcomes take too lengthy). This is particularly necessary in evaluation that requires batch dimension 1, equivalent to video.
The Groq ecosystem additionally implies that distribution throughout many TSPs merely scales out inferences per second, with a number of Q100 components beneath the identical algorithm all implementing the identical deterministic efficiency.
Jonathan acknowledged to us, as the corporate has acknowledged up to now, that Groq as an organization was constructed on a compiler-first method. Historically this form of method places numerous stress on the compiler doing the optimization (equivalent to Itanium and different VLIW processors), and sometimes results in considerations concerning the product as a complete. However, we have been advised that the group by no means touched any silicon design till six months into the software program and compiler work, permitting the corporate to lock down the important thing elements of the key ML frameworks earlier than even designing the silicon.
As a part of its funding efforts, Groq reached out to us for a corporation replace. All of Groq’s {hardware} and software program work to this point has been achieved by way of two rounds of VC funding, totaling $67.3m, with about $50m getting used to date. In that capital they’ve designed, constructed, and deployed the Q100 TSP to virtually a dozen prospects, together with the audio/visible business, datacenter, and authorities labs. The second technology product can be effectively underway. This newest Series C funding spherical of $300m, led by Tiger Global Management and D1 Capital, will enable the corporate to develop from 120 individuals to 250 by the top of the yr, help present and future prospects with greater groups, and allow a progressive roadmap.
Groq acknowledged in our briefing that its second technology product will construct on its distinctive design factors, providing options for patrons that have been excited by Q100 however produce other necessities for his or her workloads. Each technology of Groq’s TSP, based on the corporate, can have half a dozen distinctive promoting factors out there (some public, some not), with one purpose at the very least to displace as many GPUs as doable with a single TSP to be able to give prospects the perfect TCO.
