Single page Print

Our testing methods
We tested using our tried-and-true "inside the second" methods. Since we don't have FCAT equipment up here at TR North, we used Fraps to generate all our performance numbers.

Fraps gives us information about things happening at the start of the rendering pipeline—not, as FCAT does, at the end of the pipeline, when frames reach the display. Having both sets of numbers would be better, but the Fraps data is largely sufficient for the kind of testing we're doing here. We don't expect there to be much of a discrepancy between Fraps and FCAT numbers on single-GPU, single-monitor configurations like these.

This time, we've run most of our Fraps numbers through a three-frame low-pass filter. This filter is designed to compensate for one of the side effects of triple buffering. It should smooth out irregularities in our frame time measurements that don't actually affect when the frames are shown on the display. We didn't apply the filter to our BioShock Infinite numbers, since that game is based on the Unreal Engine. Most UE games don't use triple buffering, so the filter isn't appropriate for them.

Whether filtered or not, our "inside the second" Fraps numbers are far more informative than the raw frames-per-second data produced by more conventional benchmarking techniques. Such data can cover up problems like latency spikes and micro-stuttering, which have a real, palpable impact on gameplay.

For more information about Fraps, FCAT, and our inside-the-second methodology, be sure to read Scott's articles on the subject: Inside the second: A new look at game benchmarking and Inside the second with Nvidia's frame capture tools.

As ever, we did our best to deliver clean benchmark numbers. Tests were run at least three times, and we reported the median results. Our test systems were configured like so:

Processor Intel Core i7-3770K
Motherboard Gigabyte Z77X-UD3H
North bridge Intel Z77 Express
South bridge
Memory size 4GB (4 DIMMs)
Memory type AMD Memory & Kingston HyperX
DDR3 SDRAM at 1600MHz
Memory timings 9-9-9-28
Chipset drivers INF update
Rapid Storage Technology 11.6
Audio Integrated Via audio
with drivers
Hard drive Crucial m4 256GB
Power supply Corsair HX750W 750W
OS Windows 8 Professional x64 Edition

Driver revision Base GPU
AMD Radeon R7 260X Catalyst 13.11 beta V9 1100 1625 2048
Asus Radeon R9 270 Catalyst 13.11 beta V9 975 1400 2048
XFX Radeon HD 7850 2GB Catalyst 13.11 beta V9 860 1200 2048
XFX Radeon HD 7870 Catalyst 13.11 beta V9 1000 1200 2048
Asus GeForce GTX 650 Ti Boost 2GB GeForce 331.40 beta 1020 1502 2048
Asus GeForce GTX 660 GeForce 331.40 beta 980 1502 2048

Thanks to AMD, Corsair, Crucial, and Kingston for helping to outfit our test rig. AMD, Asus, Nvidia, and XFX have our gratitude, as well, for supplying the various graphics cards we tested.

Image quality settings for the graphics cards were left at the control panel defaults, except on the Radeon cards, where surface format optimizations were disabled and the tessellation mode was set to "use application settings." Vertical refresh sync (vsync) was disabled for all tests.

We used the following test applications:

Some further notes on our methods:

  • We used the Fraps utility to record frame rates while playing 60- or 90-second sequences from the game. Although capturing frame rates while playing isn't precisely repeatable, we tried to make each run as similar as possible to all of the others. We tested each Fraps sequence three times per video card in order to compensate for variability. We've included frame-by-frame results from Fraps for each game, and in those plots, you're seeing the results from a single, representative pass through the test sequence.

  • We measured total system power consumption at the wall socket using a P3 Kill A Watt digital power meter. The monitor was plugged into a separate outlet, so its power draw was not part of our measurement. The cards were plugged into a motherboard on an open test bench.

    The idle measurements were taken at the Windows desktop with the Aero theme enabled. The cards were tested under load running Crysis 3 at the same quality settings used for our performance testing.

  • We measured noise levels on our test system, sitting on an open test bench, using a TES-52 digital sound level meter. The meter was held approximately 8" from the test system at a height even with the top of the video card.

    You can think of these noise level measurements much like our system power consumption tests, because the entire systems' noise levels were measured. Of course, noise levels will vary greatly in the real world along with the acoustic properties of the PC enclosure used, whether the enclosure provides adequate cooling to avoid a card's highest fan speeds, placement of the enclosure in the room, and a whole range of other variables. These results should give a reasonably good picture of comparative fan noise, though.

  • We used GPU-Z to log GPU temperatures during our load testing.

The tests and methods we employ are generally publicly available and reproducible. If you have questions about our methods, hit our forums to talk with us about them.