Page 2 of 2

Re: Task Scheduling, PPID, forking, NUMA

Posted: Sun Jun 30, 2019 11:05 am
by chuckula
Point 1: Oh so you were running "numad" in the background? NICE TRY making us all run in circles while you pulled strings in the background ;-)

Point 2: On your performance "clustering" issue, there are plenty of possible culprits but here's a simple setting that might help: Make sure your CPU governor is set to "performance" instead of "ondemand" or "powersave". Most Linux distros default to "ondemand" to save a little power, but in your case you don't want all that extra logic complicating things. I'm not promising it will fix all your problems but at a minimum it shouldn't hurt.

Another point that's maybe so basic we haven't brought it up: What kernel version are you running? I highly recommend a very new kernel for a Threadripper since you can't expect perfection from a scheduler that was written before AMD threw the 2990 over the wall.

Re: Task Scheduling, PPID, forking, NUMA

Posted: Mon Jul 01, 2019 8:44 am
by blastdoor
As I continue to work on better understanding the performance characteristics of the 2990wx and how best to use this thing, I'm wondering if an article written on the subject might be of interest?

I'd be happy to donate such an article to the TR cause, if anybody thinks it might be of interest/value.

Re: Task Scheduling, PPID, forking, NUMA

Posted: Mon Jul 01, 2019 8:57 am
by just brew it!
I would read it.

Re: Task Scheduling, PPID, forking, NUMA

Posted: Mon Jul 01, 2019 9:06 am
by blastdoor
chuckula wrote:
Point 1: Oh so you were running "numad" in the background? NICE TRY making us all run in circles while you pulled strings in the background ;-)

Point 2: On your performance "clustering" issue, there are plenty of possible culprits but here's a simple setting that might help: Make sure your CPU governor is set to "performance" instead of "ondemand" or "powersave". Most Linux distros default to "ondemand" to save a little power, but in your case you don't want all that extra logic complicating things. I'm not promising it will fix all your problems but at a minimum it shouldn't hurt.

Another point that's maybe so basic we haven't brought it up: What kernel version are you running? I highly recommend a very new kernel for a Threadripper since you can't expect perfection from a scheduler that was written before AMD threw the 2990 over the wall.


Re point 1 --- sorry guys! I fully confess to massive stupidity on this one. It's a good example of why people like me ought to stick to Macs ;-)

Re point 2 --- thanks for the tip, I'll look into that!

Re point 2.5 -- 5.0.6-050006-generic

Re: Task Scheduling, PPID, forking, NUMA

Posted: Mon Jul 01, 2019 2:21 pm
by blastdoor
btw, Chuck -- I just ran a little comparison of my Threadripper system to my 2014 iMac (4 GHz 4790K)... unless I'm doing something stupid (which we all know is highly likely), this comparison of a 2018 Threadripper to a 2014 Haswell should make you very happy. It makes me wonder if the R packages I'm using for this particular task are using AVX2.

Re: Task Scheduling, PPID, forking, NUMA

Posted: Mon Jul 01, 2019 2:55 pm
by chuckula
blastdoor wrote:
btw, Chuck -- I just ran a little comparison of my Threadripper system to my 2014 iMac (4 GHz 4790K)... unless I'm doing something stupid (which we all know is highly likely), this comparison of a 2018 Threadripper to a 2014 Haswell should make you very happy. It makes me wonder if the R packages I'm using for this particular task are using AVX2.


This is the part where blastdoor realizes that those wackos at Intel didn't just dump all that effort into AVX for chuckula's forum posts. (But AVX-512 was 100% for the forum posts, let's be real here)

Re: Task Scheduling, PPID, forking, NUMA

Posted: Mon Jul 01, 2019 8:32 pm
by blastdoor
chuckula wrote:
blastdoor wrote:
btw, Chuck -- I just ran a little comparison of my Threadripper system to my 2014 iMac (4 GHz 4790K)... unless I'm doing something stupid (which we all know is highly likely), this comparison of a 2018 Threadripper to a 2014 Haswell should make you very happy. It makes me wonder if the R packages I'm using for this particular task are using AVX2.


This is the part where blastdoor realizes that those wackos at Intel didn't just dump all that effort into AVX for chuckula's forum posts. (But AVX-512 was 100% for the forum posts, let's be real here)


I guess I have a chip on my shoulder regarding vector units from altivec G4 days. I know they can make a big difference *sometimes* but I take it as a bad sign when people place a lot of emphasis on it. But maybe that’s not a fair comparison— the G4 was an unbalanced design. Intels designs are very well balanced.

Re: Task Scheduling, PPID, forking, NUMA

Posted: Mon Jul 01, 2019 10:38 pm
by just brew it!
Yeah, this ain't your father's vector unit. :wink:

Now that AVX and AVX2 are widely available, with AVX-512 to become mainstream eventually as well, leveraging it where appropriate should become the norm.