The OpenPOWER Saga Continues: Can You Get POWER Inside 1U?

Back in April 2016, Tyan has introduced its IBM POWER8-based 1U servers at the OpenPOWER Summit in San Jose, California. The server market is dominated by five OEMs: HPE, Dell, IBM, Lenovo, and Cisco, with ODM Quanta and HPC specialist Supermicro being the challengers. So the fact that Tyan also sells POWER8 servers can hardly be considered a game changing fact. However, Tyan does punch well above its weight. Note that there are only four OpenPOWER POWER8 platforms out there:

1. The Wistron’s Polaris was the basis for the heavy duty S822LC (see our review here). An improved version, the Polaris Plus was the first Power8 + P100 NVLink, probably the fastest 2U HPC server available (see here).
2. The Supermicro “Briggs” which is found in the latest IBM S822LC “For Big Data”.
3. The Rackspace “Barreleye”
4. The Tyan Habanero.

As far as we know, the Supermicro board is only available as an IBM server.

The Rackspace Barreleye is the odd one out: it is 1.25 U (instead of 2U) high, 21 inches wide, and needs the power shelves of “OpenRack”. The cool thing is that it is no longer a prototype; it is a server that can actually be bought. This means that those who want to start over and make use of the many advantages of OCP have a real choice between the Intel Xeon and IBM POWER. Such servers – even in low quantities – are available from Penguin Computing (Magna 1015), Mark III systems, and Stack Velocity.

Of course, most datacenters are not switching to OCP racks (yet). If you want a standard 19 inch wide OpenPOWER server, there is a pretty good chance your server is based upon the Tyan Habanero. Habanero was not only the platform in most of the Tyan POWER8 offerings, but it is also the board that can be found inside the IBM S812LC and the Penguin Computing Magna 2001. Of course, the OpenPOWER server market is still a young and small market, but Tyan did have quite a bit of impact.

So when we saw that Tyan made a 1U server based upon this Habanero platform, that caught our eye. The power-hungry POWER8 inside a density optimized form factor? And they feed it with a PSU of “only” 750 W? Remember, most POWER8 servers come with 1000+ Watt PSUs.

Now at this point you may be asking: yet another article about OpenPOWER? Isn’t all effort going to waste as IBM’s piece of the server market is shrinking and Intel reigns supreme? Why would IBM be able to succeed where so many others have failed?

We are among the first to recognize that Intel still rules the server world, and that there is no alternative to the midrange Xeon E5 when you want the best performance-per-watt available. Furthermore the arrival of servers based upon AMD’s Naples CPU will put even more pressure on all non-x86 server CPU alternatives, as it is expected to ship with up to 32 cores, 64 thread, offer 8 DDR4 channels and integrate no less than 128 PCIe lanes. And in the long-run (and perhaps most importantly), it will force Intel to come up with better performance/watt/dollar offerings, ratcheting up the pressure on non-x86 server CPUs even further.

As important as performance per watt is, several markets – HPC, Analytics, and AI chief among them – consider performance the most important metric. Wattage has to be kept under control, but that is it. Who cares whether the CPU is consuming 200W or 120W when it is placed in a machine with a Terabyte of RAM and two 300W GPUs?

The fact that Rackspace and Google have invested quite a bit in the “Barreleye” and will continue to do so is a good sign (there is a POWER9 Barreleye) but not a guarantee. Both companies and all large datacenters still rely on Intel Xeons for the foreseeable future. The experiment can quickly be terminated.

However, the announcement that China’s largest Internet portal – Tencent – will also be using OpenPOWER servers is another sign that the OpenPOWER technology convinces people that it is a viable alternative. At first, the deal was just to license the affordable IBM S822LC (made by Supermicro) to the Chinese reseller Inspur, and Inspur – being a “local” reseller – would sell it to Tencent. However, this deal spurred Inspur to begin developing their own OpenPOWER products. Consequently, OpenPOWER is allowing IBM and partners to break into the fastest growing server market: China. The openness of the software and hardware ecosystem and the strong performance of the CPU cores puts OpenPOWER is a very unique position in China compared to both Intel and ARM. That is a very solid business plan if you ask us.

Getting to the meat of today’s article, we have the Tyan GT75-BP012. Anton has already described the Tyan GT75 servers in great detail here, so we will recap and add a few details.

The Tyan GT75 machines (just like the Tyan TN71-BP012 servers launched a year ago) are based on one IBM POWER8 Turismo Single Chip Module (SCM) processor, offering either eight or ten cores. This CPU finds itself paired with Tyan’s Habanero motherboard, the same as in IBM’s most affordable OpenPOWER server, the S812LC.

The board has 32 DIMM slots using four IBM Centaur memory buffer chips (MBCs). Since the operational voltage of the Centaur chip PHY maxes out at 1.43V, only low power DDR3 DIMMs are supported. The largest supported DIMMs are the quad ranked 32 GB DIMMs with 4 Gbit chips, allowing the server to have up to 1 TB of RAM. Unfortunately, the latest 8 Gbit based DIMMs are not supported. Tyan ships the server with eight 16 GB DIMMs – for a total of 128GB – if you take the standard configuration.

As the OpenPOWER POWER8 has to fit and operate within a 1U home, the clockspeed is limited to 2.328 GHz nominal. However, that is just a paper spec just like the clockspeed of the Xeon E5. In reality, the power governor defaults to “on demand”. In that case, the CPU runs at 2.06 GHz at low load, and boost up to 3.025 GHz when the CPU is fully loaded. The speedsteps are very small, only +/- 30 MHz, so the second highest speedstep is 2.99 GHz. Below you find the configuration table of all Tyan GT75 servers.

In today’s article we’re review the basic model, the BSP012G75V4H-B4C. Notice the twelve (!) fans.

The Tyan GT75-BP012 makes use of Tyan’s mezzanine cards for networking and for the storage controller. As a result, it can equipped with up to four 3.5” hot-swappable SATA 6G HDD/SSDs and four network controllers (1 GbE or 10 GbE) without using the 8-lane PCIe riser.

Now if you’ve been counting the CPIe lanes required for all of this, it seems like we should be a bit short, and indeed that’s the case. Digging a bit deeper, we’ll find that the server is using a PLX PEX8748 PCIe switch to take a PCIe 3.0 x8 root port from the CPU and switch it among the LAN riser, SATA riser, and the two black PCIe x8 slots.

Our testing was conducted on Ubuntu Server 16.04 (kernel 4.2.0) with gcc compiler version 5.2.1. As we were only able to get everything working appropriately with that specific software combination, we were not able to use something newer.

Last but not least, we want to note how the performance graphs have been color-coded. Orange is for used for POWER8 CPUs, while the latest generation of the Intel Xeon (v4) gets dark blue, the previous one (v3) gets light blue, and older Xeon generations are colored with the default gray.

Tyan GT75-BSP012 (1U)

The Tyan GT75-BSP012 is based up on Tyan’s “Habanero” platform.

IBM S812LC (2U)

The IBM S812LC is also based up on Tyan’s “Habanero” platform. The board inside the IBM server is thus designed by Tyan.

Intel’s Xeon E5 Server – S2600WT (2U Chassis)

Our trusty Xeon E5 collection includes the E5-2699 v4, E5-2699v3, and E5-2690.

All C-states are enabled in the BIOS.

SuperMicro 6027R-73DARF (2U Chassis)