Why does Windows bounce a single-threaded program around?

Monopoly money comes in many flavors: 7, Vista, XP, 2K, ME, 98, etc.

Moderators: Flying Fox, Ryu Connor

Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 1:58 pm

Surely it'd be better to keep the single threaded program (ffmpeg2theora) on the same core all the time so you don't have to waste intercore bandwidth copying/syncing L2/L1 caches. I understand video encoding doesn't benefit too much from caches because the data just flows in and isn't used again, but still, why does WIndows pass the single thread around to every CPU in my Phenom II X6? Instead, I have to set the process's affinity to run on one CPU... surely I shouldn't have to do this all the time?
Mothership: Thuban 1055T@3.7GHz, 12GB DDR3, M5A99X EVO, GTX470+Icy Vision Rev.2@840/3800, Vertex 2E 60GB
Supply ship: Sargas@2.8GHz, 12GB DDR3, M4A88TD-V EVO/USB3
Corsair: Macbook Air Ivy Bridge
Crayon Shin Chan
Minister of Gerbil Affairs
 
Posts: 2246
Joined: Fri Sep 06, 2002 11:14 am
Location: Malaysia

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 2:08 pm

Have you profiled the application to see if it really benefits from fixing affinity? Most of the time the difference should be minimal but video encoders you may indeed see a benefit.
Image
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
Flying Fox
Gerbil God
 
Posts: 24489
Joined: Mon May 24, 2004 2:19 am

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 2:11 pm

I haven't compiled it myself. I think you need to compile it with profiling hooks to profile a binary, right?
RIght now it seems it has 23 threads, but only one is obviously doing any work that's bottlenecked by the CPU, because when I set the affinity to more than 1 core, the CPU usage for the core that was running it drops.
Mothership: Thuban 1055T@3.7GHz, 12GB DDR3, M5A99X EVO, GTX470+Icy Vision Rev.2@840/3800, Vertex 2E 60GB
Supply ship: Sargas@2.8GHz, 12GB DDR3, M4A88TD-V EVO/USB3
Corsair: Macbook Air Ivy Bridge
Crayon Shin Chan
Minister of Gerbil Affairs
 
Posts: 2246
Joined: Fri Sep 06, 2002 11:14 am
Location: Malaysia

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 2:15 pm

Even if the data was used more than once, the time required to sync the caches is typically going to be tiny compared to the time between context switches.

As a guess, what's happening here is that the core the thread was originally using happens to be busy doing something else at the instant when your application wakes up from an I/O wait. Rather than stall the application that just woke up (or interrupting the other application), it makes more sense to reschedule it to another core.

Could the scheduler do a better job of keeping threads on the same core? Yeah, maybe. But I doubt you'd notice much (if any) difference in practice.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37830
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 6:29 pm

Crayon Shin Chan wrote:I haven't compiled it myself. I think you need to compile it with profiling hooks to profile a binary, right?
RIght now it seems it has 23 threads, but only one is obviously doing any work that's bottlenecked by the CPU, because when I set the affinity to more than 1 core, the CPU usage for the core that was running it drops.

Well, it can be as simple as stop-watching an encode job with and without affinity to see which 1 is faster.
Image
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
Flying Fox
Gerbil God
 
Posts: 24489
Joined: Mon May 24, 2004 2:19 am

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 6:33 pm

Crayon Shin Chan wrote:Surely it'd be better to keep the single threaded program (ffmpeg2theora) on the same core all the time so you don't have to waste intercore bandwidth copying/syncing L2/L1 caches. I understand video encoding doesn't benefit too much from caches because the data just flows in and isn't used again, but still, why does WIndows pass the single thread around to every CPU in my Phenom II X6? Instead, I have to set the process's affinity to run on one CPU... surely I shouldn't have to do this all the time?

Keep in mind that even if it shifts cores every couple minutes, it has executed literally billions of instructions on the previous core before it shifted. It's not like Windows shifts it around every ten instructions or anything.

At the time the program shifts to a new core, it could be that Windows chose to run something else on that core immediately previous to that thread getting a time slice, and when its turn came up, the Windows kernel decided to move the program rather than whatever else just got started on that core.

It's not really something that's going to have a measurable impact unless the program is shifting cores many times per second. Even a second on a 2GHz core is 2 billion instruction cycles.
Buub
Maximum Gerbil
Silver subscriber
 
 
Posts: 4203
Joined: Sat Nov 09, 2002 11:59 pm
Location: Seattle, WA

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 6:56 pm

I vote for testing it out. The real problem is you're going to have to do multiple runs of very large files to ensure smaller differences will be noticed, and have repeatable activity to potentially interfere. I suppose you could always slow the clock too, possibly exaggerating the effects.
Looking for Knowledge wrote:When drunk.....
I want to have sex, but find I am more likely to be shot down than when I am sober.
Heiwashin
Grand Gerbil Poohbah
 
Posts: 3027
Joined: Wed Dec 13, 2006 1:21 pm
Location: Denham Springs, LA

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 6:58 pm

Crayon Shin Chan wrote:Surely it'd be better to keep the single threaded program (ffmpeg2theora) on the same core all the time so you don't have to waste intercore bandwidth copying/syncing L2/L1 caches. I understand video encoding doesn't benefit too much from caches because the data just flows in and isn't used again, but still, why does WIndows pass the single thread around to every CPU in my Phenom II X6? Instead, I have to set the process's affinity to run on one CPU... surely I shouldn't have to do this all the time?


Because CPU scheduling on most OSes, Windows included, is simple? It is a balancing act between the general case and the special case. For best average throughput of a process with multiple processes running, you want the process to be able to switch execution units as necessary. As you note, for a single CPU bound process on an idle system, running on the same execution unit provides the best throughput.

So, how do you figure out on the fly which case you are in? How do you tell which processes will suffer the most from waiting to run on the same core they last ran on? Windows is a desktop OS, so to ensure perceived responsiveness of the OS, it is best that a process run again as soon as possible, even if that is less efficient overall. Turning on processor affinity lets you override this behavior when you know best and allows the OS scheduler to remain simple and effective for the average case. Linux allows you, at least now, to use entirely different scheduling algorithms to fit the workload you are giving the machine, I don't know if Windows has some funky registry tuning parameters that do similar things, though I tend to doubt it.

--SS
SecretSquirrel
Gerbil Jedi
Gold subscriber
 
 
Posts: 1719
Joined: Tue Jan 01, 2002 7:00 pm
Location: The Colony, TX (Dallas suburb)

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 7:05 pm

Come to think of it, would raising the processing priority take care of that problem?
Looking for Knowledge wrote:When drunk.....
I want to have sex, but find I am more likely to be shot down than when I am sober.
Heiwashin
Grand Gerbil Poohbah
 
Posts: 3027
Joined: Wed Dec 13, 2006 1:21 pm
Location: Denham Springs, LA

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 8:38 pm

Heiwashin wrote:Come to think of it, would raising the processing priority take care of that problem?

No. The process is still going to give up the CPU whenever it needs to wait for I/O.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37830
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 10:36 pm

You didn't mention which version of Windows you were using. I figure a link to some SMP performance/scaling for Windows might assuage your performance concerns.

Multicore performance under Vista and Vista R2 massively out scales XP.
"Welcome back my friends to the show that never ends. We're so glad you could attend. Come inside! Come inside!"
Ryu Connor
Global Moderator
Gold subscriber
 
 
Posts: 3558
Joined: Thu Dec 27, 2001 7:00 pm
Location: Marietta, GA

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 11:26 pm

Heiwashin wrote:Come to think of it, would raising the processing priority take care of that problem?

It's not a problem, that's my point! This stuff is very highly tuned by the Windows kernel team. It wouldn't do this if it was a problem.
Buub
Maximum Gerbil
Silver subscriber
 
 
Posts: 4203
Joined: Sat Nov 09, 2002 11:59 pm
Location: Seattle, WA

Re: Why does Windows bounce a single-threaded program around?

Postposted on Mon Nov 01, 2010 11:31 pm

Buub wrote:
Heiwashin wrote:Come to think of it, would raising the processing priority take care of that problem?

It's not a problem, that's my point! This stuff is very highly tuned by the Windows kernel team. It wouldn't do this if it was a problem.

Yup, and "it's not a problem" is the point I was trying to make (unsuccessfully, I guess) in my first reply.
(this space intentionally left blank)
just brew it!
Administrator
Gold subscriber
 
 
Posts: 37830
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Why does Windows bounce a single-threaded program around?

Postposted on Tue Nov 02, 2010 12:34 am

just brew it! wrote:
Buub wrote:
Heiwashin wrote:Come to think of it, would raising the processing priority take care of that problem?

It's not a problem, that's my point! This stuff is very highly tuned by the Windows kernel team. It wouldn't do this if it was a problem.

Yup, and "it's not a problem" is the point I was trying to make (unsuccessfully, I guess) in my first reply.

Yeah you just said the same thing in more engineery-speak! :-)
Buub
Maximum Gerbil
Silver subscriber
 
 
Posts: 4203
Joined: Sat Nov 09, 2002 11:59 pm
Location: Seattle, WA

Re: Why does Windows bounce a single-threaded program around?

Postposted on Tue Nov 02, 2010 5:17 am

OK cool, I'll leave it alone then.
Mothership: Thuban 1055T@3.7GHz, 12GB DDR3, M5A99X EVO, GTX470+Icy Vision Rev.2@840/3800, Vertex 2E 60GB
Supply ship: Sargas@2.8GHz, 12GB DDR3, M4A88TD-V EVO/USB3
Corsair: Macbook Air Ivy Bridge
Crayon Shin Chan
Minister of Gerbil Affairs
 
Posts: 2246
Joined: Fri Sep 06, 2002 11:14 am
Location: Malaysia


Return to Windows

Who is online

Users browsing this forum: No registered users and 9 guests