First up is my favorite game of the year so far, Borderlands 2. The shoot-n-loot formula of this FPS-RPG mash-up is ridiculously addictive, and the second installment in the series has some of the best writing and voice acting around.
Our game benchmarking methods are different from what you'll find elsewhere in part because they're based on chunks of gameplay, not just scripted sequences. Below is a look at our 90-second path through the "Opportunity" level in Borderlands 2.
As you'll note, this session involves lots of fighting, so it's not exactly repeatable from one test run to the next. However, we took the same path and fought the same basic contingent of foes each time through. The results were pretty consistent from one run to the next, and final numbers we've reported are the medians from five test runs.
We used the game's highest image quality settings at the 27" Korean monitor resolution of 2560x1440.
You can click the buttons above to switch between the results for the Radeon and the GeForce. Please note that we have only one set of results for the GeForce, and those are from Windows 8. The plots above show frame rendering times in milliseconds, so lower times are better.
As you can see, the plots for the 7950 in Win7 and Win8 look to be very similar, with little spikes to 40 ms or so throughout our test session. By contrast, the plot for the GTX 660 Ti is smoother; the frame latencies are generally lower, without so many high-latency frames interspersed throughout.
The GeForce is a little faster in the traditional FPS average, although the Radeons still reach over 60 FPS, which is usually considered sufficient.
The performance difference between the two cards becomes clearer in the latency-focused 99th percentile frame time. This number is simply the threshold below which 99% of all frames were rendered. The GTX 660 Ti produces all but the final 1% of frames in under 19.6 milliseconds—equivalent to about 50 FPS—while the Radeon's threshold is over 30 milliseconds—or about 30 FPS.
The 99th percentile frame time is just one point on a curve, and here's the curve. As you can see, the Radeon's performance tracks with the GeForce's until we reach the last 7% or so of the frames rendered. Those higher-latency frames are where the Radeon trips up, and from the plots, we know they're distributed throughout the test session. Interestingly enough, the Radeon appears to perform marginally better in Windows 8, although the differences are pretty minor.
Our final latency-sensitive metric singles out those frames that take an especially long time to render. The goal of this measurement is to get a sense of "badness," of the severity of any slowdowns encountered during the test session. What we do is add up any time spend rendering beyond a threshold of 50 milliseconds. (Frame times of 50 ms are equivalent to a frame rate of 20 FPS, which is awfully slow.) For instance, if a frame takes 70 milliseconds to render, it will contribute 20 milliseconds to our "badness" index. The higher this index goes, the more time we've spent waiting on especially high-latency frames, and the less fluid the game animation has been.
Again here, the 7950 falls behind the GeForce GTX 660 Ti, and going back to Windows 7 isn't any help. That said, the Radeon doesn't perform terribly; it only spends a small amount of time working on frames beyond our 50-ms threshold, so in-game animations should be relatively fluid. In fact, that's what we experienced during our testing sessions. The game was playable enough on the Radeon, under both versions of Windows. The thing is, playing on the GeForce was simply a little bit smoother, both subjectively and by the numbers.
Why is that? For more insight into exactly what's going on, we can zoom in on a small portion of our test run and look at the individual frame times. This is just a tiny bit of our overall data set, but it does help illustrate things. Click through the buttons to see each config's results.
The 7950 suffers from these momentary hiccups. They're not terribly dramatic, but still, the GeForce avoids them. Often, the long frame times are succeeded by several shorter frame times, as if the system is waiting on some long operation to complete the first frame and is ready with portions of the others once it finishes.
The presence of little hiccups of this size isn't a big deal on a small scale, but it becomes more of an issue once we know that those longer frames comprise roughly 5-7% of the total frames produced, distributed throughout the test session. Given the size of the latency spikes here, they're still not really a big problem, but they do bring the Radeon down—and make the GeForce look (and feel) like the superior solution.