I think it's awesome. It's the kind of setup that we all wish we could pull off with a DIY SFF box (Xeon notwithstanding). The only real limitation seems to be the number of DIMM slots being half of what the CPU natively supports. Other than that, well, AMD GPU's. Full GK110's would be faster and more flexible.
On Mac, I think there's better OpenCL support on the AMD chips? Maybe the nvidia chips were too hot though I would think they'd have better pro application support. Some people are complaining that the second GPU is likely going to be unused most of the time. Not that it really matters, Apple's pretty far behind on OpenGL support, what are they, on version 2.1? Maybe 3? I think 4.2 is the current one? Plus, any pros with a large investment in PCIe devices or internal storage are not looking forward to having to buy a PCIe external box to hook up to the machine via thunderbolt. Or a thunderbolt-based RAID for the drives they used to have internal.
The traditional Apple pro user, from what I gather, was hoping for an updated dual-socket Xeon workstation with internal PCIe and SATA slots so they could just keep on truckin'. The new Mac Pro is very interesting and it will be interesting to see how well it gets adopted or if it's the thing that finally pushes Apple pros into looking at Windows based workflows and alternative programs.
Could be better OpenCL support- but that's really up to Apple to make happen, and that may be why they went with AMD, as AMD has the hardware and was likely more willing to work with them to get the driver architecture where Apple needed it. It definitely makes sense from that perspective- though the Nvidia hardware would have been more powerful, and I can't ignore the gaming angle here, where Nvidia's SLI drivers are far more mature
I'm sure Mac Pro users were looking forward to a dual-socket system; the old Mac Pro design really isn't that 'old', since it could easily be upgraded to support Ivy-E, while giving users everything they need, like multiple PCIe 3.0 x16 slots and any number of SATA/SAS drive bays. This new system is fairly compact- so you have to wonder if there isn't more coming from Apple to satisfy those needs.
And if you think about it- what are you going to do on a Mac that you would need all of that processing power, that wouldn't be better done on a Linux cluster? I'd think that Apple decided to put in a second GPU instead of a second CPU because the things that Macs are used for and excel at are better suited to GPGPU processing rather than relatively slow CPU processing. It's not like a six or eight core Sandy/Ivy wouldn't be enough to keep a pair of top-end GPUs fed with compute work
Lastly, I don't think we can count Apple out for different designs; even this design could be turned into a 'square' by adding another side to accommodate a second CPU, and it'd be nothing to make it a few inches taller to support six or eight 2.5" (or even 3.5") SATA/SAS bays for storage. I think that it's far less likely for Apple to go this route, yes, but it's not like they couldn't.