TR Forums

Hey guys, just thought I'd link to Extremetech's report on IBM's new Power8. I think TR needs to cover this too, seeing as this is a major development in the world of computing, something we don't see everyday.

Wow, that thing's a monster!

TDP of 250 watts?! Would like to see the chassis that this gets deployed in, hah!

Yeah, it's a monster, alright. IBM wants to be the ARM of servers, as the article mentions. It's interesting to note that while ARM is planning to make a go for it in the server space, IBM's already well-positioned in that space and may actually stand a better chance of succeeding. Intel's got their hands full right now after beating AMD to a pulp. Say, am I the only guy who thinks AMD should also try its hand in the Power space? I know they're practically penniless these days but hey, they may be able to pull it off. Plus, they have a long history with IBM, trading tech as well as their very employees, who seem to be like electrons zipping in and out between the likes of AMD, Samsung, Nvidia, and IBM (and probably Intel too).They should really focus on ARM and something real to sell in the server space if they're really bent on ditching x86. They've helped Intel enough already, and have lost enough blood too doing it.

Oi. Will the cooling be liquid or evaporative?

Folks were water-cooling big iron 40+ years ago.

JustAnEngineer wrote:

Folks were water-cooling big iron 40+ years ago.

Like the Crays with the wrap-around seats. I was wondering if Power8 had moved beyond the limits of water cooling and needed direct refrigeration/evaporative cooling.

ronch wrote:

Say, am I the only guy who thinks AMD should also try its hand in the Power space? I know they're practically penniless these days but hey, they may be able to pull it off. Plus, they have a long history with IBM, trading tech as well as their very employees, who seem to be like electrons zipping in and out between the likes of AMD, Samsung, Nvidia, and IBM (and probably Intel too).They should really focus on ARM and something real to sell in the server space if they're really bent on ditching x86. They've helped Intel enough already, and have lost enough blood too doing it.

While the idea of hedging their bets by putting a foot in each camp has some merit, I think attempting ARM and POWER at the same time would spread them too thin at a time when they really need to focus if they want to survive.

Going the POWER route has the advantage of being a SOI-based design, and AMD probably has the most experience with SOI outside of IBM; but they'd be competing head-to-head with IBM in the server space. That doesn't sound like a winning strategy to me.

AMD's Hypertransport expertise and experience working with vendors of massively parallel systems (Cray) uniquely positions them to pull off a strategy where they harness massive farms of ARM cores to do something useful in the server space. IMO if they need to pick one, ARM is probably it. It's still a long shot, but less of a long shot than POWER.

Captain Ned wrote:

JustAnEngineer wrote:

Folks were water-cooling big iron 40+ years ago.

Like the Crays with the wrap-around seats. I was wondering if Power8 had moved beyond the limits of water cooling and needed direct refrigeration/evaporative cooling.

Water cooling will probably be good enough. Based on TDP figures alone these aren't much more power-hungry than AMD's FX-9590 chips, which are OK with common liquid cooling solutions.

just brew it! wrote:

ronch wrote:

Say, am I the only guy who thinks AMD should also try its hand in the Power space? I know they're practically penniless these days but hey, they may be able to pull it off. Plus, they have a long history with IBM, trading tech as well as their very employees, who seem to be like electrons zipping in and out between the likes of AMD, Samsung, Nvidia, and IBM (and probably Intel too).They should really focus on ARM and something real to sell in the server space if they're really bent on ditching x86. They've helped Intel enough already, and have lost enough blood too doing it.

While the idea of hedging their bets by putting a foot in each camp has some merit, I think attempting ARM and POWER at the same time would spread them too thin at a time when they really need to focus if they want to survive.

Going the POWER route has the advantage of being a SOI-based design, and AMD probably has the most experience with SOI outside of IBM; but they'd be competing head-to-head with IBM in the server space. That doesn't sound like a winning strategy to me.

AMD's Hypertransport expertise and experience working with vendors of massively parallel systems (Cray) uniquely positions them to pull off a strategy where they harness massive farms of ARM cores to do something useful in the server space. IMO if they need to pick one, ARM is probably it. It's still a long shot, but less of a long shot than POWER.

You're probably right. With their bank account they just can't afford to fight too many battles. Just something that popped in my head though.

Interesting. 24 cores/192 threads and 1 TB of RAM at the high end, at least to start - compared with up to 256 POWER7 cores/1024 threads and 16 TB of RAM in a p795. Wonder if IBM plans on scaling up, and hence this is just the "early" high end option - I'd be very surprised if not.

No great surprise, though, to see IBM open up - it'd be damn hard for any single company to match Intel in CPU R&D these days. I do wish them luck: the thought of a CPU monoculture (or even just two options - ARM and x86) scares the hell out of me.

I was just reading about it at the Linley Group, which is where the famous David Kanter is now writing - it goes into much more detail over here. 32 64bit channels of RAM - they couldn't run that many lines out of the package, so they made an intermediate buffer that communicates serially with the CPU to handle the RAM access instead! I love overengineered stuff like this.
http://www.linleygroup.com/mpr/article.php?id=11088

I ran a POWER4+ workstation (9114-275, Intellistation POWER 275) back in the day - it was very limited. Heavy and loud, there wasn't much it could do that other cheaper computers couldn't already do. Its only saving grace was that it had an exotic CPU, a plastic door with a cool always-on LCD, took 10 minutes to POST, and came with a special muffler to hide the noise it made. So I really hope POWER8 gains traction and more software/driver support, so I can run that exotic CPU again for a practical purpose.

Crayon Shin Chan wrote:

So I really hope POWER8 gains traction and more software/driver support, so I can run that exotic CPU again for a practical purpose.

Well, it sounds like IBM has partnered with Canonical on the software front, so if things go according to plan there should be a viable Linux ecosystem for it.

TwistedKestrel wrote:

Wow, that thing's a monster!

TDP of 250 watts?! Would like to see the chassis that this gets deployed in, hah!

IBM z Series (http://www-03.ibm.com/systems/resources ... 50x450.JPG) It's the size of two refrigerators.

ronch wrote:

Yeah, it's a monster, alright. IBM wants to be the ARM of servers, as the article mentions. It's interesting to note that while ARM is planning to make a go for it in the server space, IBM's already well-positioned in that space and may actually stand a better chance of succeeding. Say, am I the only guy who thinks AMD should also try its hand in the Power space? They should really focus on ARM and something real to sell in the server space if they're really bent on ditching x86.

IBM doesn't want to have to move to a new CPU architecture. At this point, IBM is the only one driving Power ISA development. Everyone has moved on to ARM, and the Chinese like MIPS. Power is the odd man out, so it's more desperation to keep Power alive more then anything.

This is ten years too late. They're only doing this because no one cares about Power anymore, and there isn't any money in it for them. ARM has taken over the niche of the mainstream RISC ISA, which Power used to fill, and ARM has all of the momentum and mindshare.

AMD has enough problems. They don't need to jump onto a non-existent bandwagon.

just brew it! wrote:

Crayon Shin Chan wrote:

So I really hope POWER8 gains traction and more software/driver support, so I can run that exotic CPU again for a practical purpose.

Well, it sounds like IBM has partnered with Canonical on the software front, so if things go according to plan there should be a viable Linux ecosystem for it.

There is also Yellow Dog Linux which has produced a RHEL based PPC distro since 1999. (http://www.fixstars.com/en/technologies/linux/)

Does Power8 run ESX and Hyper-V?

I only deal with things in my immediate scope which means Windows and Linux servers running on x86-64.
ARM, RISC, ASIC - I have no idea what sort of SMB sofware would work or require these architectures.
I only use Xeons and Opterons at the moment, though I'm keeping an eye on the new Avoton 8C Atoms.

Chrispy_ wrote:

Does Power8 run ESX and Hyper-V?

I only deal with things in my immediate scope which means Windows and Linux servers running on x86-64.
ARM, RISC, ASIC - I have no idea what sort of SMB sofware would work or require these architectures.
I only use Xeons and Opterons at the moment, though I'm keeping an eye on the new Avoton 8C Atoms.

You won't find ESX or Hyper-V running on Power hardware. IBM has their PowerVM suite of virtualization technologies which tightly integrate with the Power processors. You can run several of IBM's own OSes (AIX, i aka OS/400, etc) and several flavors of Linux (RHEL, Ubuntu, SLES).

Seems like a good time to disclose I work for IBM, though in the GTS division (aka infrastructure services) and not STG (aka hardware).

I don't see how the extra power is gonna be an issue for the Power 8, the chips themselves are about 50% larger in area than Xeons and are designed for servers so I doubt air cooling is going to have any issues here. They may have to designate a larger socket clearance, but that's it.

slowriot wrote:

You won't find ESX or Hyper-V running on Power hardware. IBM has their PowerVM suite of virtualization technologies which tightly integrate with the Power processors. You can run several of IBM's own OSes (AIX, i aka OS/400, etc) and several flavors of Linux (RHEL, Ubuntu, SLES).

Does this mean they can't run Microsoft servers as VM's, or just that they're unsuitable from a value perspective?

Even if they could, it would be horribly inefficient since they would need to emulate the entire x86 instruction set in software.

Chrispy_ wrote:

Does this mean they can't run Microsoft servers as VM's, or just that they're unsuitable from a value perspective?

It's a no go. As JBI points out it would require emulating x86 in software just for starters.

You won't need ESXi on POWER. I remember playing with this feature called LPARs - basically you can split your computer up to behave like two or more separate computers, each running a different OS. This was back in 2002, before virtualization. And they all have direct hardware access, no VT-d-like extensions needed. This should replace ESXi (it's even lower level to boot), and it felt less like a layer on top of PC hardware and more like it was part of the hardware, as if it was normal to run your hardware that way.

Crayon Shin Chan wrote:

This was back in 2002, before virtualization.

More like 1972

http://en.wikipedia.org/wiki/VM_(operating_system)

I'm not so worried about the hypervisor IBM use to create VM's or hardware divide, I'm more interested in what guest OSes you can actually run on these chips;

Running 50 VM's on a 32-thread Xeon 2P server, for example, can be done with ESX if you want Linux/Windows and Hyper-V if you just want Windows. The businesses I'm involved with are largely windows-based and even companies that are bigger than everything I deal with combined could/would still have at least 50% Windows environments.

I must be phrasing the question too indirectly, let me retry:
Is there any point getting excited about these new chips if you need Windows Server (representative of a majority of all companies) or is this niche hardware for datacenter/specialised environments?

The article is headlined with "Intel’s x86 server monopoly" so if these can't run the most popular x86 software on the market (Windows) then what's the point?

Chrispy_ wrote:

I'm not so worried about the hypervisor IBM use to create VM's or hardware divide, I'm more interested in what guest OSes you can actually run on these chips

Anything that has been ported to POWER architecture.

Chrispy_ wrote:

I must be phrasing the question too indirectly, let me retry:
Is there any point getting excited about these new chips if you need Windows Server (representative of a majority of all companies) or is this niche hardware for datacenter/specialised environments?

The article is headlined with "Intel’s x86 server monopoly" so if these can't run the most popular x86 software on the market (Windows) then what's the point?

I agree that trying to push something like this in the desktop space (where Windows still dominates) would be silly; but many of the largest consumers of compute cycles out there (Google and Facebook to name a couple...) run their infrastructure on Linux. So yes, this is aimed squarely at enterprise/datacenter (and probably HPC as well) users who are hardware agnostic, by virtue of the fact that they are *not* running Windows.

Power efficiency -- not x86 compatibility -- is the overriding concern when you start talking about deploying hundreds of thousands of servers. If Power8 can beat x86 on performance per watt IBM will have customers for these chips.

It should also be noted that Google is apparently a charter member of the OpenPOWER Foundation, so they are already interested in seeing where this goes.

Edit: Maybe the issue here is how you're parsing the headline? "Intel's x86 server monopoly" in this context means "Intel's server monopoly (which they have managed to do with x86)" not "Intel's monopoly on x86-based servers".

just brew it! wrote:

Power efficiency -- not x86 compatibility -- is the overriding concern when you start talking about deploying hundreds of thousands of servers. If Power8 can beat x86 on performance per watt IBM will have customers for these chips.

I agree. But, what if one has to rethink then recode their job's work-flow to exploit the efficiency? Hundreds of threads in one address space is different than 4 or 16. For very large complex applications, the cost of software, including development, test, maintenance, often far outstrips the hardware cost. Not to mention that programmers who can grok that level of scalability cost more than those who can't.

Personally, I think the Power ISA and memory system architecture is a thing of beauty to the low-level programmer. But, these days, very few people write low-level code (assembler or C) to the point where that matters anywhere near as much as it used to.

Chrispy_ wrote:

I must be phrasing the question too indirectly, let me retry:
Is there any point getting excited about these new chips if you need Windows Server (representative of a majority of all companies) or is this niche hardware for datacenter/specialised environments?

The article is headlined with "Intel’s x86 server monopoly" so if these can't run the most popular x86 software on the market (Windows) then what's the point?

Okay. A bit of basic information (apologies if I cover ground you already know; I want to make sure that everything pertinent is covered off.)

If you buy a computer today, it can have CPUs running one of five (or six, depending how you count) different instruction set architectures: x86/64; ARM; Itanium; SPARC; or POWER. (Yes, okay, MIPS is still out there, but it's pretty much embedded applications only now.) The latter three are predominantly used in the Unix space: HP-UX (Itanium; also the legacy NonstopOS and VMS); Solaris; and AIX. POWER is more familiar to most people in its PowerPC form: same instruction set, just a different target market (it wasn't always so, mind; there were differences between POWER and PowerPC in the early days, but they did converge eventually.)

Will Windows run on these chips? Short answer: no. Long answer: Windows NT used to run on PowerPC systems (back in version 3.51 and 4.0 days), so Windows 2012 could, in theory, be ported without difficulty to POWER - but I really don't see Microsoft putting the effort in; the market would be way too small (never mind the software that would never be ported over and would have to be run in emulation - if it could be made to run at all.)

Linux, on the other hand - that's a first class operating system; one of three that IBM actively supports on POWER systems (the other two being AIX and IBM i.) And there's an emulation layer - the same technology that Apple licensed as Rosetta - that lets x86 Linux software run reasonably quickly on a POWER Linux LPAR.

So why should Windows sysadmins care? Competition. If POWER, and the operating systems it runs, can do the job that you need Windows for, it gives you a bargaining chip that can drive down the cost of whichever platform you end up using. "Why should I pay twenty million dollars for this x86 solution when I can pay IBM ten million dollars for something that does the same job?" sort of thing.

There's also a degree of scale and capability that POWER has that x86, at the very top end, still can't easily match. Niche, sure, and for a majority of applications, you don't need that capability, but when you need it, you really do need it. I currently work in a banking environment, and they have several p795 frames installed to handle certain workloads.

In the end, speaking as a sysadmin, I like having multiple options open to me. Even if I do end up managing a monoculture of Windows hosts (that would be the day I start looking for openings at my local lawn mowing firm...), I like knowing that there are other places I can turn to if options A or B turn out to not be suitable (or start trying to turn the screws on software maintenance or similar.)

There's also the point that IBM has been in the virtualisation game for decades, long before VMware hit the market. Theo de Raadt has had a great deal to say about x86 virtualisation, not much of it complimentary; I would trust PowerVM a lot more than most x86 virtualisation platforms when it comes to security (not that I have much say in the matter, but still.)

Ta. That makes a bit more sense.

Power8 is completely useless to me/my companies directly, but will impact the market in a way that might drive Intel/Microsoft prices down eventually.

MarkG509 wrote:

just brew it! wrote:

Power efficiency -- not x86 compatibility -- is the overriding concern when you start talking about deploying hundreds of thousands of servers. If Power8 can beat x86 on performance per watt IBM will have customers for these chips.

I agree. But, what if one has to rethink then recode their job's work-flow to exploit the efficiency? Hundreds of threads in one address space is different than 4 or 16. For very large complex applications, the cost of software, including development, test, maintenance, often far outstrips the hardware cost. Not to mention that programmers who can grok that level of scalability cost more than those who can't.

"Big data" customers (like the ones I already mentioned) and HPC don't care for the most part. Their developers understand how to scale things; their applications are already coded to run on huge farms of loosely interconnected CPUs and/or are "embarrassingly parallel" (thousands or millions of threads crunching independent streams of data). Both of these types of application are almost trivially easy to port to an architecture with fewer physical CPUs having more threads on each, provided you have the I/O bandwidth to support the application (and IBM really knows how to do I/O).

Furthermore, no re-coding at the application level is needed if you are using these multi-core beasts to consolidate physical servers via virtualization.

I wish IBM would partner up with AMD for the x86 market and make 22nm SOI cpu chips for AMD.

A 22nm high end AMD CPU would breath a little life into AMD with a improved memory controller and other improvements like a shorter execution pipeline.

Chrispy_ wrote:

Ta. That makes a bit more sense.

Power8 is completely useless to me/my companies directly, but will impact the market in a way that might drive Intel/Microsoft prices down eventually.

Just out of curiosity here...

So even if the most cost-effective solution to a specific subset of your workload involved running some sort of back-end server on a POWER based system, you would dismiss it out-of-hand because it does not run Windows?

vargis14 wrote:

I wish IBM would partner up with AMD for the x86 market and make 22nm SOI cpu chips for AMD.

A 22nm high end AMD CPU would breath a little life into AMD with a improved memory controller and other improvements like a shorter execution pipeline.

Moving to smaller process tech doesn't automatically make your memory controller better or shorten your execution pipeline. Unless you redesign the chip (and I really don't see IBM redesigning AMD's chips for them) all it will do is reduce your power consumption, and maybe give you a little more clock speed headroom (don't get me wrong, these improvements would be worthwhile too).