Personal computing discussed

Moderators: renee, Flying Fox, morphine

 
chuckula
Minister of Gerbil Affairs
Topic Author
Posts: 2109
Joined: Wed Jan 23, 2008 9:18 pm
Location: Probably where I don't belong.

Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 8:07 am

Servethehome has an interesting preliminary test that pits a dual-socket 18 core Xeon 6150 Gold against a dual-socket 32 core Epyc 7601.

A bunch of the noise about Xeon vs. Epyc has been a comparison of the highest-end chips, but frankly I think in most real-world server workloads the Epyc is really competing against the Xeon Gold series and not the platinum series.

The interesting takeaway is that in the compiler benchmarks the Xeon Gold 6150 basically ties the Epyc 7601 even though the 7601 system has 77% more cores. More interestingly is that far from being massively expensive, the Xeon Gold 6150 -- at least at list price, which we all know is not the real price most people pay -- is noticeably less expensive than the 7601.

The compiler test results in particular are markedly different than the rather rushed results that Anandtech posted last month. I know that Anand did some quick testing of Xeon and Epyc that got some play (and was advertised by AMD) but frankly I'm not impressed with their methods for several reasons including that they apparently didn't setup their Xeon box correctly and were using a non-production Epyc system that was customized by AMD just for their review. I generally trust STH more than Anandtech in this area since Anandtech is really more a consumer-focused site.
4770K @ 4.7 GHz; 32GB DDR3-2133; Officially RX-560... that's right AMD you shills!; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
 
derFunkenstein
Gerbil God
Posts: 25427
Joined: Fri Feb 21, 2003 9:13 pm
Location: Comin' to you directly from the Mothership

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 8:23 am

How do the EPYC dies talk to each other? I wonder if stuff is getting bounced not just between CCXes (which is already problematic) but between CPU dies. The next test in the STH article is c-ray and AMD performs about like you'd expect, besting everything with gusto.
I do not understand what I do. For what I want to do I do not do, but what I hate I do.
Twittering away the day at @TVsBen
 
chuckula
Minister of Gerbil Affairs
Topic Author
Posts: 2109
Joined: Wed Jan 23, 2008 9:18 pm
Location: Probably where I don't belong.

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 8:28 am

derFunkenstein wrote:
How do the EPYC dies talk to each other? I wonder if stuff is getting bounced not just between CCXes (which is already problematic) but between CPU dies. The next test in the STH article is c-ray and AMD performs about like you'd expect, besting everything with gusto.


The Epyc chips talk to each other using Infinity Fabric that is physically implemented using 64 lanes of PCIe connectivity.
That's why you will see a dual-socket Epyc System that has pretty much the same number of available PCIe lanes as a single socket system because half of the PCIe lanes on each chip are devoted to talking to the other chip.

Servethehome had an interesting deep-dive on the matter here: https://www.servethehome.com/amd-epyc-a ... deep-dive/
4770K @ 4.7 GHz; 32GB DDR3-2133; Officially RX-560... that's right AMD you shills!; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
 
derFunkenstein
Gerbil God
Posts: 25427
Joined: Fri Feb 21, 2003 9:13 pm
Location: Comin' to you directly from the Mothership

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 11:15 am

OK I didn't want to say it was over PCIe if it wasn't, but I had surmised that it had. Is that also how the cores on a single package talk to each other?

I guess what I'm getting at is that we know AMD's inter-module connectivity is a bottleneck, so my guess is that the compiler benchmark is getting stuck on it. But other tasks that let the CPU cores do their own thing without depending on each other shows that EPYC can do well in certain workloads. c-ray, for example.
I do not understand what I do. For what I want to do I do not do, but what I hate I do.
Twittering away the day at @TVsBen
 
chuckula
Minister of Gerbil Affairs
Topic Author
Posts: 2109
Joined: Wed Jan 23, 2008 9:18 pm
Location: Probably where I don't belong.

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 11:44 am

Yeah, if you leave the core to its own devices with a synthetic test like C-Ray that fits almost entirely into L1 cache (forget about L2) then it scales pretty much by core count, especially since C-Ray isn't really using the more advanced vector units to do ray tracing.

However, C-Ray isn't what people buy these servers for.

The compiler benchmark isn't the only benchmark that matters but it is a pretty good real-world test of how the CPUs scale under a complex workload that actually has to move some data around from time to time. Compared to a full-bore database the compiler runs are actually far less chatty since most of the compilation outside of certain serialized sections like the linker can occur independently on each core, but there is still quite a bit more data moving around than in an all-cache benchmark. Incidentally, the compilers don't use AVX either so no inherent advantage for Intel based on that aspect of the chip design.
4770K @ 4.7 GHz; 32GB DDR3-2133; Officially RX-560... that's right AMD you shills!; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
 
Waco
Maximum Gerbil
Posts: 4850
Joined: Tue Jan 20, 2009 4:14 pm
Location: Los Alamos, NM

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 11:57 am

Compiler benchmarks beyond a certain number of cores don't scale well due to other dependencies unless very carefully designed make files are in use. Do we know if other higher-core count systems scale well in said benchmark?
Victory requires no explanation. Defeat allows none.
 
chuckula
Minister of Gerbil Affairs
Topic Author
Posts: 2109
Joined: Wed Jan 23, 2008 9:18 pm
Location: Probably where I don't belong.

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 12:02 pm

Waco wrote:
Compiler benchmarks beyond a certain number of cores don't scale well due to other dependencies unless very carefully designed make files are in use. Do we know if other higher-core count systems scale well in said benchmark?


They had some very early benchmarks of the 8176 and 8180 in both dual and quad socket systems here: https://www.servethehome.com/quad-intel ... enchmarks/

Warning: Not necessarily apples-to-apples with the first linked story.

In general there definitely is scaling with CPU core count all the way out to quad-socket servers. Definitely not perfectly linear but it's not like the quad-socket servers hit a brick wall and are only as fast as the dual-socket setups.
4770K @ 4.7 GHz; 32GB DDR3-2133; Officially RX-560... that's right AMD you shills!; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
 
Flying Fox
Gerbil God
Posts: 25690
Joined: Mon May 24, 2004 2:19 am
Contact:

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 1:21 pm

Database and virtualization benchmarks would be the key use cases for these?
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
 
derFunkenstein
Gerbil God
Posts: 25427
Joined: Fri Feb 21, 2003 9:13 pm
Location: Comin' to you directly from the Mothership

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 1:30 pm

chuckula wrote:
Yeah, if you leave the core to its own devices with a synthetic test like C-Ray that fits almost entirely into L1 cache (forget about L2) then it scales pretty much by core count, especially since C-Ray isn't really using the more advanced vector units to do ray tracing.

However, C-Ray isn't what people buy these servers for.

The compiler benchmark isn't the only benchmark that matters but it is a pretty good real-world test of how the CPUs scale under a complex workload that actually has to move some data around from time to time. Compared to a full-bore database the compiler runs are actually far less chatty since most of the compilation outside of certain serialized sections like the linker can occur independently on each core, but there is still quite a bit more data moving around than in an all-cache benchmark. Incidentally, the compilers don't use AVX either so no inherent advantage for Intel based on that aspect of the chip design.

The problem is these are the only two tests that STH ran on the EPYC system. I'm sure when they get one of their own that'll change.
I do not understand what I do. For what I want to do I do not do, but what I hate I do.
Twittering away the day at @TVsBen
 
chuckula
Minister of Gerbil Affairs
Topic Author
Posts: 2109
Joined: Wed Jan 23, 2008 9:18 pm
Location: Probably where I don't belong.

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Aug 01, 2017 1:46 pm

derFunkenstein wrote:
chuckula wrote:
Yeah, if you leave the core to its own devices with a synthetic test like C-Ray that fits almost entirely into L1 cache (forget about L2) then it scales pretty much by core count, especially since C-Ray isn't really using the more advanced vector units to do ray tracing.

However, C-Ray isn't what people buy these servers for.

The compiler benchmark isn't the only benchmark that matters but it is a pretty good real-world test of how the CPUs scale under a complex workload that actually has to move some data around from time to time. Compared to a full-bore database the compiler runs are actually far less chatty since most of the compilation outside of certain serialized sections like the linker can occur independently on each core, but there is still quite a bit more data moving around than in an all-cache benchmark. Incidentally, the compilers don't use AVX either so no inherent advantage for Intel based on that aspect of the chip design.

The problem is these are the only two tests that STH ran on the EPYC system. I'm sure when they get one of their own that'll change.


They already have one of their own and they have several articles about the teething problems it has been experiencing (which is why some of those benchmarks aren't fully published yet).

For example, those compiler benchmarks for Epyc are more than twice as fast as what STH got when they first tested the server. https://www.servethehome.com/amd-epyc-v ... july-2017/
4770K @ 4.7 GHz; 32GB DDR3-2133; Officially RX-560... that's right AMD you shills!; 512GB 840 Pro (2x); Fractal Define XL-R2; NZXT Kraken-X60
--Many thanks to the TR Forum for advice in getting it built.
 
derFunkenstein
Gerbil God
Posts: 25427
Joined: Fri Feb 21, 2003 9:13 pm
Location: Comin' to you directly from the Mothership

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Sep 19, 2017 6:36 pm

https://www.servethehome.com/amd-epyc-7 ... nd-review/

Here’s some more fun stuff. Some of these tests existed when the thread was created but there’s a new mountain of stuff.

AVX-512 is obviously still Intel’s domain so it may age better but lots of AMD wins in terms of per-dollar and overall better performance.

Edit: it’s not the top-end parts btw. This is $1000 Epyc vs $1000 worth of Xeon, max.
I do not understand what I do. For what I want to do I do not do, but what I hate I do.
Twittering away the day at @TVsBen
 
DragonDaddyBear
Gerbil Elite
Posts: 985
Joined: Fri Jan 30, 2009 8:01 am

Re: Interesting preliminary Xeon & Epyc Benchmarks

Tue Sep 19, 2017 9:10 pm

Phoronix has had a lot of benchmarks on the Linux side. It's really interesting to see how well his Epyc sample is doing compared to the dual Xenon Golds he has, especially when you look at the price. Overall I think Epyc is a win for AMD in the use cases it does well, which is more often than I expected.

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On