Our testing methods
We tested using our tried-and-true "inside the second" methods. Since we don't have FCAT equipment up here at TR North, we used Fraps to generate all our performance numbers.
Fraps gives us information about things happening at the start of the rendering pipeline—not, as FCAT does, at the end of the pipeline, when frames reach the display. Having both sets of numbers would be better, but the Fraps data is largely sufficient for the kind of testing we're doing here. We don't expect there to be much of a discrepancy between Fraps and FCAT numbers on single-GPU, single-monitor configurations like these.
This time, we've run most of our Fraps numbers through a three-frame low-pass filter. This filter is designed to compensate for one of the side effects of triple buffering. It should smooth out irregularities in our frame time measurements that don't actually affect when the frames are shown on the display. We didn't apply the filter to our BioShock Infinite numbers, since that game is based on the Unreal Engine. Most UE games don't use triple buffering, so the filter isn't appropriate for them.
Whether filtered or not, our "inside the second" Fraps numbers are far more informative than the raw frames-per-second data produced by more conventional benchmarking techniques. Such data can cover up problems like latency spikes and micro-stuttering, which have a real, palpable impact on gameplay.
For more information about Fraps, FCAT, and our inside-the-second methodology, be sure to read Scott's articles on the subject: Inside the second: A new look at game benchmarking and Inside the second with Nvidia's frame capture tools.
As ever, we did our best to deliver clean benchmark numbers. Tests were run at least three times, and we reported the median results. Our test systems were configured like so:
|Processor||Intel Core i7-3770K|
|North bridge||Intel Z77 Express|
|Memory size||4GB (4 DIMMs)|
|Memory type||AMD Memory & Kingston HyperX
DDR3 SDRAM at 1600MHz
|Chipset drivers||INF update 18.104.22.1681
Rapid Storage Technology 11.6
|Audio||Integrated Via audio
with 6.0.01.10800 drivers
|Hard drive||Crucial m4 256GB|
|Power supply||Corsair HX750W 750W|
|OS||Windows 8 Professional x64 Edition|
|Driver revision||Base GPU
|AMD Radeon R7 260X||Catalyst 13.11 beta V9||1100||1625||2048|
|Asus Radeon R9 270||Catalyst 13.11 beta V9||975||1400||2048|
|XFX Radeon HD 7850 2GB||Catalyst 13.11 beta V9||860||1200||2048|
|XFX Radeon HD 7870||Catalyst 13.11 beta V9||1000||1200||2048|
|Asus GeForce GTX 650 Ti Boost 2GB||GeForce 331.40 beta||1020||1502||2048|
|Asus GeForce GTX 660||GeForce 331.40 beta||980||1502||2048|
Thanks to AMD, Corsair, Crucial, and Kingston for helping to outfit our test rig. AMD, Asus, Nvidia, and XFX have our gratitude, as well, for supplying the various graphics cards we tested.
Image quality settings for the graphics cards were left at the control panel defaults, except on the Radeon cards, where surface format optimizations were disabled and the tessellation mode was set to "use application settings." Vertical refresh sync (vsync) was disabled for all tests.
We used the following test applications:
Some further notes on our methods:
We used the Fraps utility to record frame rates while playing 60- or 90-second sequences from the game. Although capturing frame rates while playing isn't precisely repeatable, we tried to make each run as similar as possible to all of the others. We tested each Fraps sequence three times per video card in order to compensate for variability. We've included frame-by-frame results from Fraps for each game, and in those plots, you're seeing the results from a single, representative pass through the test sequence.
We measured total system power consumption at the wall socket using a P3 Kill A Watt digital power meter. The monitor was plugged into a separate outlet, so its power draw was not part of our measurement. The cards were plugged into a motherboard on an open test bench.
The idle measurements were taken at the Windows desktop with the Aero theme enabled. The cards were tested under load running Crysis 3 at the same quality settings used for our performance testing.
We measured noise levels on our test system, sitting on an open test bench, using a TES-52 digital sound level meter. The meter was held approximately 8" from the test system at a height even with the top of the video card.
You can think of these noise level measurements much like our system power consumption tests, because the entire systems' noise levels were measured. Of course, noise levels will vary greatly in the real world along with the acoustic properties of the PC enclosure used, whether the enclosure provides adequate cooling to avoid a card's highest fan speeds, placement of the enclosure in the room, and a whole range of other variables. These results should give a reasonably good picture of comparative fan noise, though.
We used GPU-Z to log GPU temperatures during our load testing.
The tests and methods we employ are generally publicly available and reproducible. If you have questions about our methods, hit our forums to talk with us about them.