What do your resources look like (CPU/mem)?
Since you're streaming, could be a network bandwidth issue (if not your end, then theirs?)?
one core is pegged out, the other is 70 or 80%, memory I don't remember, but it's not maxed out. Unless it take over a 10mbit connection... anyways that'd be a buffer problem and wouldn't play I believe.
Silverlight 4 doesn't support hardware acceleration video decoding. Silverlight 5 Beta does support that feature, however the Netflix player has not yet implemented this. You can try downloading the Silverlight 5 Beta though and see if it helps.
I've look at the silverlight task manger thing (alt + ctrl + shift + m) and it says it's using hardware acceleration and using only 70%ish cpu I know.
I'll try Silverlight 5 beta tonight, thanks for the tip