Personal computing discussed
puppetworx wrote:TR should be including more multiplayer testing in their CPU tests
puppetworx wrote:Of course it's impossible to fully control all the variables in a multiplayer test
JohnC wrote:unless used with a specific benchmarking utility
Damage wrote:You cited some old-ish results from us.
Damage wrote:But yeah, I wish I had the time and resources to do proper mega-multiplayer testing in properly in something like BF3. Maybe I can take a crack at it at some point soon. Maybe with BF4..?
End User wrote:Another CPU related topic that has been discussed is what influence the next gen consoles will have on PC gaming. Will the new consoles and their multi-threaded capabilities spike CPU usage in games across the board? Will 4 Intel cores be enough for some PC games this time next year?
Damage wrote:But yeah, I wish I had the time and resources to do proper mega-multiplayer testing in properly in something like BF3. Maybe I can take a crack at it at some point soon. Maybe with BF4..?
JohnC wrote:Yea, here's a thought: games are a poor "benchmarking" tool unless used with a specific benchmarking utility which can use same (the most demanding) exact in-game level with same exact in-game actions during the same exact time period. If every reviewer just logs into the game and just randomly hops around at a random in-game location - you'll get such inconsistently useless results as in your links. That Russian tester himself admits that the results might not be reliable because he used a multplayer portion (64-player server on Kiasar Railroad) for testing, which is even worse than using a singleplayer portion of the game.
Edit: same goes for the fact that game developers tend to release updates from time to time which may have significant effect on performance.
JohnC wrote:That Russian tester himself admits that the results might not be reliable because he used a multplayer portion (64-player server on Kiasar Railroad) for testing, which is even worse than using a singleplayer portion of the game.
End User wrote:A discussion in another thread got me thinking about how TR benchmarks CPU performance in games.
These result don't appear to reflect real world gaming performance whereas these results do (full post). Thoughts?
NovusBogus wrote:The problem with any benchmark is it will necessarily favor certain features, subroutines, architectures, whatever because that's what happens to be the bottleneck for the task at hand. The trick to benchmarks is to do a whole lot of them, as diverse as possible, so that things average out. I'd love to see TR benchmarks get a bit more exotic with less focus on mainstream games simply because mainstream games have a long history of being "optimized" for AMD or nVidia in exchange for a fat wad of cash.
Krogoth wrote:The vast majority of games only take advantage of two threads and a vanishing small number only take advantage of four threads.
NovusBogus wrote:The problem with any benchmark is it will necessarily favor certain features, subroutines, architectures, whatever because that's what happens to be the bottleneck for the task at hand. The trick to benchmarks is to do a whole lot of them, as diverse as possible, so that things average out. I'd love to see TR benchmarks get a bit more exotic with less focus on mainstream games simply because mainstream games have a long history of being "optimized" for AMD or nVidia in exchange for a fat wad of cash.
NovusBogus wrote:I'd love to see TR benchmarks get a bit more exotic with less focus on mainstream games simply because mainstream games have a long history of being "optimized" for AMD or nVidia in exchange for a fat wad of cash.
Star Brood wrote:I've yet to see a StarCraft 2 CPU test with 4x4 multiplayer with motherships involved. That will stress even a theoretical 10GHz i7 4960K.
StarCraft 2 tests are very easy to reproduce as well, thanks to replay functionality. You simply follow a specific player camera around the map.
Damage wrote:Hmm. I don't read Russian and haven't slogged through the article carefully. Working today, so I don't really have time. I'll offer a few responses though.
It's true our test results in the particular case you linked came from the single-player game and not a multiplayer match with 64 people. We've documented how we tested--and how we test, which continues to evolve. (You cited some old-ish results from us.) These two test scenarios in BF3 are different, but both of them happen in the real world. I don't think one is more "real" than the other, although the multiplayer test may be more relevant to some folks.
I'm happy to see folks doing the hard work of multiplayer testing with lots of players on one server. That is a very tough thing to do properly. Since the play sessions aren't repeatable at all, you'd need to do lots of sampling in order to get good results. We've not been able to dedicate the time, along with the time of 63 very dedicated friends, to making it happen.
However, I'm a little sad to see such hard work boiled down to an FPS average as the end result. Doesn't really tell you what you need to know about comparative gaming smoothness. I think we offer more granularity with our frame-by-frame plots, our latency-focused metrics like "time beyond 50 ms," and with our latest addition: slow-mo videos, like we've used in our recent GPU reviews:
http://techreport.com/review/25167/fram ... on-hd-7990
We've not done smoothness videos in CPU reviews, but we could consider it in the future.
Are those BF3 results more "real" than our old BF3 results? Nope. Are they more relvant for BF3 multiplayer? Prolly. Are they helpful? Depends on whether they sampled sufficiently while testing, in part. Still, not as much as they could be, when you have only FPS avg/min as the result.
End User wrote:Another CPU related topic that has been discussed is what influence the next gen consoles will have on PC gaming. Will the new consoles and their multi-threaded capabilities spike CPU usage in games across the board? Will 4 Intel cores be enough for some PC games this time next year?