Ah, it’s the eternal battle, the unending duopoly duel: GeForce versus Radeon, Radeon versus GeForce. The skirmishes are ongoing, but the victor is never decided for long. Today, another chapter in the story unfolds.
Doesn’t seem that long ago, back in August, when the GeForce GTX 660 Ti first hit the scene and squared off against the Radeon HD 7950. That match-up ushered in a new generation of competition among ridiculously powerful video cards at around 300 bucks. Nvidia had the advantage going in, since it was facing off against an already established competitor; it knew what the GTX 660 Ti had to do in order to win. AMD, however, has been unusually feisty lately, and it had other ideas. At the last minute, the Radeon team rolled out a new BIOS that added dynamic clock speeds to the 7950. The result was an incredibly slight win for the GTX 660 Ti on points, but in the end, we threw our hands up and said the differences mattered little.
We’re vaguely astonished by how much things have changed since then.
Of course, we have a new crop of games for the holiday season, headlined by titles like Borderlands 2, Hitman: Absolution, Sleeping Dogs, and Assassin’s Creed III. AMD’s newfound aggressiveness means many of these games are part of its Gaming Evolved program, so they should run very well on Radeon graphics cards—and maybe, you know, not so well on those pesky GeForces.
In fact, accentuating its stronger ties to game developers, AMD has taken to bundling a trio of these games with its Radeon HD 7950 cards. Cramming that sort of gaming goodness into the box with a graphics card certainly changes the value equation.
As if that weren’t enough, AMD has also released Catalyst 12.11 beta drivers that promise a roughly 15% across-the-board performance increase for its 7000-series Radeons. New drivers often bring performance gains for individual games, but general improvements of that magnitude are uncommon. AMD tells us it has employed new insights in tuning its relatively young GCN architecture.
What’s more, Windows 8 is out, and we’ve transitioned our test rigs to the new operating system.
Add up all of these changes, and you have a recipe for realignment in ongoing battle for GPU supremacy. Are we still at rough parity, or have AMD’s bold moves allowed it to push into the lead? We’ve deployed our infamous “inside the second” testing methods with a host of the latest games in order to find out.
Our lovely contestants
Pictured above is the Sapphire HD 7950 Vapor-X, our representative from the Radeon camp for this little hoedown. The 7950 Vapor-X is our first look at a retail product with the new Boost BIOS, and it ups the ante by sporting a peak Boost clock of 950MHz, 25MHz above stock. Sapphire’s Vapor-X cooler sprouts quad heatpipes that snake into a large array of cooling fins situated beneath dual fans. The shroud that covers the whole assembly may be the finest expression of the F-117 Stealth fighter look that has rampaged through the enthusiast PC hardware scene in recent years. Although it sticks out maybe a quarter-inch beyond the 10.5″ length of the card itself, there’s no way that thing shows up on radar.
The HD 7950 Vapor-X sells for $329.99 at Newegg and comes with a bunch of inducements to buy, including copies of Sleeping Dogs, Hitman: Absolution, and Far Cry 3, along with a 20%-off coupon for Medal of Honor: Warfighter. There’s also a $20 mail-in rebate attached right now. If you buy two, AMD CEO Rory Read will come to your house and personally serenade you from outside of your window. I hear he has quite the voice.
Looking over the listings at Newegg, 7950 cards are going for as little as $299.99. However, only a few other cards can match the Vapor-X’s 950MHz boost clock, and they all cost more than the Sapphire.
We’ve pitted the HD 7950 Vapor-X against our returning champ from the GeForce side, the Zotac GTX 660 Ti AMP. This baby sports Zotac’s charming “angry bumblebee” look but is scaled down massively from its GTX 670 and 680 brethren. The card is only 6.75″ long, giving it a distinctive miniature vibe we like to call “low BOM cost chic.”
There’s something to be said for keeping costs down, though. The GTX 660 Ti AMP! is currently going for $299.99, even though it’s a hot-clocked card. In fact, the Zotac’s boost frequency is one of the highest among the GTX 660 Ti cards available. Also, its 6.6 GT/s memory is 10% faster than most of its competitors, even though they cost as much as 350 bucks.
In a bid not to be totally left behind by AMD’s cornucopia of bundled games, most GTX 660 Ti cards right now (including the AMP!) come with a free copy of Assassin’s Creed III. Also, Zotac currently matches Sapphire’s $20 rebate offer with its own, for those who enjoy filling out microscopic forms.
|Sapphire HD 7950 Vapor-X
|Zotac GTX 660 Ti AMP!
Although they’re positioned against each other in the market, these two cards really are somewhat different classes of hardware, as both the picture and table above illustrate. The 7950 is based on a slightly cut-down Tahiti GPU with a 384-bit memory interface. The GTX 660 Ti’s interface is half that width at 192 bits. The Radeon has the theoretical advantage in ROP rate, shader flops, and memory bandwidth—and the gap is quite large, in the last case. The 7950 has 3GB of memory, too, while the 660 Ti has 2GB. The GeForce can eclipse it only in texture filtering prowess.
The 7950 even has more appetite for power, requiring six- and eight-pin auxiliary inputs, while the GTX 660 Ti gets away with dual six-pin plugs.
Still, Nvidia’s Kepler architecture has proven to be shockingly efficient in many cases, so the GTX 660 Ti’s lower specs won’t necessarily translate into lower performance. That’s why we test these things. Speaking of which…
Our testing methods
As ever, we did our best to deliver clean benchmark numbers. Tests were run at least three times, and we’ve reported the median result.
Our test systems were configured like so:
|16GB (4 DIMMs)
DDR3 SDRAM at 1600MHz
Rapid Storage Technology Enterprise 220.127.116.111
with Realtek 18.104.22.16862 drivers
F240 240GB SATA
GTX 660 Ti AMP!
|GeForce 310.54 beta
Radeon HD 7950 Vapor-X
12.11 beta 8
Thanks to Intel, Corsair, and Gigabyte for helping to outfit our test rigs with some of the finest hardware available. AMD, Nvidia, and the makers of the various products supplied the graphics cards for testing, as well.
Unless otherwise specified, image quality settings for the graphics cards were left at the control panel defaults. Vertical refresh sync (vsync) was disabled for all tests.
In addition to the games, we used the following test applications:
Some further notes on our methods:
We used the Fraps utility to record frame rates while playing either a 60- or 90-second sequence from the game. Although capturing frame rates while playing isn’t precisely repeatable, we tried to make each run as similar as possible to all of the others. We tested each Fraps sequence five times per video card in order to counteract any variability. We’ve included frame-by-frame results from Fraps for each game, and in those plots, you’re seeing the results from a single, representative pass through the test sequence.
We measured total system power consumption at the wall socket using a Yokogawa WT210 digital power meter. The monitor was plugged into a separate outlet, so its power draw was not part of our measurement. The cards were plugged into a motherboard on an open test bench.
The idle measurements were taken at the Windows desktop with the Aero theme enabled. The cards were tested under load running Skyrim at 2560×1440 with the Ultra quality presets, 4X MSAA, and FXAA enabled.
We measured noise levels on our test system, sitting on an open test bench, using an Extech 407738 digital sound level meter. The meter was mounted on a tripod approximately 10″ from the test system at a height even with the top of the video card.
You can think of these noise level measurements much like our system power consumption tests, because the entire systems’ noise levels were measured. Of course, noise levels will vary greatly in the real world along with the acoustic properties of the PC enclosure used, whether the enclosure provides adequate cooling to avoid a card’s highest fan speeds, placement of the enclosure in the room, and a whole range of other variables. These results should give a reasonably good picture of comparative fan noise, though.
We used GPU-Z to log GPU temperatures during our load testing.
The tests and methods we employ are generally publicly available and reproducible. If you have questions about our methods, hit our forums to talk with us about them.
First up is my favorite game of the year so far, Borderlands 2. The shoot-n-loot formula of this FPS-RPG mash-up is ridiculously addictive, and the second installment in the series has some of the best writing and voice acting around.
As you may know, our game benchmarking methods are different from what you’ll find elsewhere, in part because they’re based on chunks of gameplay, not just scripted sequences. We’re trying something different this time around: embedding videos of typical gameplay sessions in the article. Below is a look at our 90-second path through the “Opportunity” level in Borderlands 2.
As you’ll note, this session involves lots of fighting, so it’s not exactly repeatable from one test run to the next. However, we took the same path and fought the same basic contingent of foes each time through. The results were pretty consistent from one run to the next, and final numbers we’ve reported are the medians from five test runs.
We used the game’s highest image quality settings at the 27″ Korean monitor resolution of 2560×1440.
Our first result is a simple plot of the time needed to render each frame during one of our test runs. Because the frame render times are reported in milliseconds, lower times are preferable. Note that, although you may see FPS-over-time plots elsewhere, those usually are based on averaging FPS over successive one-second intervals; as a result, they tend to mask momentary slowdowns almost entirely. Our plots are sourced from the raw frame time data instead.
Right away, this approach gives us some insights. The GTX 660 Ti’s frame times tend to be very low, generally under 20 ms and rarely ranging above that mark. By contrast, the Radeon HD 7950’s plot is riddled with spikes up to twice that long or longer.
A traditional FPS average doesn’t really capture the difference in how these two cards perform in Borderlands 2. Yes, the Radeon’s average is lower, but it’s still over the supposedly golden 60-FPS mark. Usually, producing an average that high would be considered quite good, but we felt the difference between the 7950 and the GTX 660 Ti clearly while testing.
We think gamers would be better served by skipping the FPS average and instead taking a latency-focused approach to frame delivery, if they really want to understand gaming performance. One alternative method is to consider the 99th percentile frame time, which is simply the threshold below which 99% of all frames have been generated. In the chart above, the Radeon HD 7950 has delivered 99% of the frames in under 31.7 milliseconds or less. That means all but the last one percent of frames were produced at a rate of 30 FPS or better—not too shabby.
Compared to the GeForce, though, the Radeon isn’t doing so well. The GeForce delivers 99% of frames in under 20 milliseconds, which is the equivalent of about 50 FPS. That’s why playing the game on the GeForce feels perceptibly smoother. I think these 99th percentile numbers more accurately convey the sense of things one gets from studying those initial frame time plots—and from playing the game on both cards.
Our 99th percentile cutoff has proven to be a pretty good choice for capturing a sense of comparative performance. However, we have to be careful, because it’s just one point along a curve. We can plot the entire frame latency curve, using data taken from all five runs, in order to get a better sense of overall performance. Over time, I’ve grown accustomed to reading these curves, and they’re now my favorite way to illustrate gaming performance.
As you can see above, the Radeon’s performance is very close to the GeForce’s much of the time. The two cards’ latency curves are similar up to about 90% of the frames rendered. Once we reach the last 10% or so, though, they begin to diverge, with the Radeon’s curve shooting upward sooner, to higher reaches. For some reason, the 7950 struggles to render a portion of the frames as quickly as its GeForce counterpart. We know from the initial frame time plots that those difficult frames are distributed throughout the test run as intermittent and fairly frequent spikes.
Our 99th percentile metric rules out the last one percent of frames, instead focusing on the general latency picture. That’s helpful, as we’ve seen, but we also want to pay attention to the worst delays, the ones that are likely to impact the fluidity of gameplay. After all, a fast computer is supposed to curtail those big slowdowns.
We can measure the “badness” of long frame times by adding up all of the time spent working on frames beyond a given threshold. In this case, we’ve picked 50 milliseconds as our cutoff. That’s equivalent to 20 FPS, and we figure any animation moving slower than 20 FPS will probably be noticeably halting and choppy. Also, 50 ms is equivalent to three vertical refresh intervals on a 60Hz display.
These results are somewhat heartening. Although the Radeon does spend twice as long above our threshold as the GeForce, neither card wastes much time at all working on especially long-latency frames. In other words, both cards offer pretty good playability in this test scenario. Subjectively, I prefer the smoother gameplay produced by the GeForce, but the Radeon doesn’t struggle too mightily. Still, the gap between them is much larger than the 64-to-72 difference in FPS averages would seem to suggest.
Guild Wars 2
Yes, yes I am attempting to benchmark an MMORPG. Guild Wars 2 has a snazzy new game engine that will stress even the latest graphics cards, and I think we can get reasonably reliable results if we’re careful. My test run consisted of a simple stroll through the countryside, which is reasonably repeatable. I didn’t join any parties, fight any bandits, or try anything elaborate like that, as you can see in the video below.
Uh oh. Once again, we’re seeing lots of spikes from the 7950, while the GeForce’s frame times are smoother and more consistent. Interestingly, though, the Radeon still produces more total frames across our test run, as you can tell from its longer line in the plot above. You know what that means?
Yep, despite all of the spikes, the Radeon’s FPS average is actually higher than the GTX 660 Ti’s. The difference is minor, but the FPS average really doesn’t tell the whole story.
Our latency-focused metrics capture the difference, though. The Radeon and GeForce perform almost identically through the majority of the latency curve, but the 7950 struggles with the last five percent of frames. Again, those problem frames are interspersed throughout the test run. The result is another big gap between the two cards’ 99th percentile frame times, again in favor of the GTX 660 Ti.
Only the 7950 registers on our “badness” meter in this test scenario. The GTX 660 Ti doesn’t take over 50 milliseconds to produce any of the frames, while the 7950 spends almost a fifth of a second crunching frames above our threshold.
Our Sleeping Dogs test scenario consisted of me driving around the game’s amazingly detailed replica of Hong Kong at night, exhibiting my terrifying thumbstick driving skills.
This game’s graphics are intensive enough that we were easily able to stress these GPUs at 1080p.
This is a closer contest than we saw in the first two games. The frame time plots for both cards show some spikes, as one might expect when the game must constantly stream in new areas of the city. The spikes are more frequent on the GeForce, but the magnitude of the spikes on the Radeon is greater. The result is a clear advantage for the 7950 in the FPS average, but only a slight edge in our latency-sensitive 99th percentile frame time metric—and more time spent by the Radeon above 50 ms.
Assassin’s Creed III
This game appears to be a thought experiment centered around what would happen if the Quaker Oats guy had invented parkour in 18th-century Boston. As you’ll see in the video below, the one thing that would not have happened is his hat falling off. That thing must be glued on there.
Since the AC3 menu doesn’t lend itself to screenshots, I’ll just tell you that we tested at 1920×1080 with environment and shadow quality set to “very high” and texture and anti-aliasing quality set to “high.” I understand that the “high” AA setting uses FXAA HQ with no multisampling. This game also supports Nvidia’s TXAA, but Nvidia has gated off access to that mode from owners of Radeons and pre-Kepler GeForces, so we couldn’t use it for comparative testing.
Well, this one is more straightforward than the others, at least. The GTX 660 Ti is much, much faster than the Radeon HD 7950 in this scenario, regardless of how you slice it.
In this game, Max Payne has sobered up and gotten a job with a shadowy government agency, yet somehow things still went totally sideways. He’s decided to stop talking about it so much, which is a relief.
I wasn’t sure how to test this game, since the object appears to be avoiding detection rather than fighting people. Do I test by standing around, observing guards’ patrolling patterns? Also, it seems that some areas of this game are much more performance-challenged than others, for reasons that aren’t entirely clear. Ultimately, I decided to test by taking a walk through Chinatown, which is teeming with people and seems to be reasonably intensive. I can’t say that good performance in this scenario would ensure solid performance in other areas of this game, though.
And we’ve finally found a good use for DX11 tessellation: bald guys’ noggins.
Yikes. This game is part of AMD’s Gaming Evolved program and is bundled with the Radeon HD 7950 right now, as is Sleeping Dogs. I had really expected better things from the Radeon as a result. You can see that the 7950’s average frame time is much lower than the GTX 660 Ti’s, but the spikes—followed by short frame times, likely due to buffering—are present throughout the test run.
The curve tells us that the high-latency frames only comprise about five percent of the total frames produced by the 7950. Still, since those long render times are present, the 7950 actually trails the GeForce GTX 660 Ti in our two latency-sensitive metrics.
Medal of Honor: Warfighter
Warfighter uses the same Frostbite 2 engine as Battlefield 3, with advanced lighting and DX11 support. That’s fortunate, because I struggled to find any other redeeming quality in this stinker of a game. Even playtesting it is infuriating. I actually liked the last Medal of Honor game, but this abomination doesn’t belong in the same series. If you enjoy on-rails shooters where it constantly feels like you’re in a tunnel, bad guys pop up randomly, and your gun doesn’t work well, then this is the game for you.
Well. Even though we have the image quality settings cranked at a resolution of 2560×1440, neither of these cards struggles in the least with the rendering workload. The result is an almost identical finish in every metric, with the slightest of advantages to the 7950 in the latency-focused numbers.
The Elder Scrolls V: Skyrim
No, Skyrim isn’t the newest game at this point, but it’s still one of the better looking PC games and remains very popular. It’s also been a particular point of focus in driver optimizations, so we figured it would be a good fit to include here.
We did, however, decide to mix things up by moving to a new test area. Instead of running around in a town, we took to the open field, taking a walk across the countryside. This change of venue provides a more taxing workload than our older tests in Whiterun.
Note: do not aggro the giants.
Not again! The Radeon HD 7950’s plot looks more like a cloud than a line, since it’s populated by a series of long and short frame times back to back, along with some nasty spikes into the 50 and 60 millisecond range.
The 7950 really doesn’t perform too poorly here—it delivers 99% of the frames in about 25 milliseconds or less, the equivalent of 40 FPS. Even if there is a lot of variance in its plot, much of it comes below that 25 ms threshold, and you’d be hard-pressed to notice it.
The more vexing problem is the series of larger spikes that happen occasionally as we tread through the countryside. Those are easily noticeable and interrupt the flow of the animation. Once again, the spikes on the GTX 660 Ti are smaller and less frequent.
Our idle power measurements demonstrate the impact of AMD’s ZeroCore Power feature, where the Radeon HD 7950 GPU powers down most of itself when the screen goes into power-save mode. When it kicks in, ZeroCore Power drops total system power consumption by 19W. Without it, when the system is idle at the desktop, the GTX 660 Ti draws a little less power than the 7950.
When running a game—Skyrim, in this case—the 7950-equipped system draws 20W more at the wall socket than the GTX 660 Ti-based test rig.
Noise levels and GPU temperatures
Both of these cards are reasonably quiet at idle, but the 7950 becomes virtually silent in ZeroCore Power mode, when its fans stop spinning. Then, only the faint whine of our slow-spinning CPU cooler generates any sound above the noise floor of our test environment.
Sapphire has tuned its Vapor-X cooler to maintain very low temperatures under load, likely in order to ensure lots of overclocking headroom for those who wish to tinker. They pay a price in additional fan noise, but that cooler is beefy enough to keep the noise levels fairly modest, regardless.
Personally, I prefer the fan profile Zotac has chosen for its GTX 660 Ti AMP!, which still keeps temperatures in check (67° C is practically cool, for a GPU) but holds down noise levels, as well. Even with its tiny cooler, the Zotac card makes less noise than the Sapphire.
This certainly isn’t the outcome we expected going into this little exercise. Given AMD’s expanded involvement with game developers and a claimed across-the-board increase in driver performance, we expected the Radeon HD 7950 to assert itself as the best choice in its class. Instead, the Radeon’s performance was hampered by delays in frame delivery across a number of games.
Our first instinct upon seeing these results was to wonder if we hadn’t somehow misconfigured our test systems or had some sort of failing hardware. We test Nvidia and AMD GPUs on separate but identical systems, so to confirm our numbers, we switched the cards between the systems and re-tested. The Radeons still exhibited the same patterns of frame latency, with no meaningful change in the results. We wondered about the possibility of a problem with our Sapphire HD 7950 Vapor-X card or its Boost BIOS causing the slowdowns, but swapping in an older, non-Boost Radeon HD 7950 card from MSI produced very similar results.
We’re also quite confident the problem isn’t confined to a single set of drivers. You see, this article has had a long and difficult history; it was initially conceived as an update comparing Catalyst 12.8 and 12.11 beta drivers. However, driver updates from AMD and Nvidia, along with some additional game releases, caused us to start testing over again last week. I can tell you that we’ve seen the same spiky frame time plots in most of these games from three separate revisions of AMD’s drivers—and, yes, Catalyst 12.11 is an improvement over 12.8, all told, even if it doesn’t resolve the latency issues.
In the end, we’re left to confront the fact that the biggest change from our prior graphics reviews was the influx of new games and new test scenarios that stress the GPUs differently than before. (The transition to Windows 8 could play some role here, but we doubt it.) For whatever reason, AMD’s combination of GPU hardware and driver software doesn’t perform as well as Nvidia’s does in this latest round of games, at least as we tested them. That’s particularly true when you focus on gameplay smoothness, as our latency-focused metrics tend to do.
Speaking of which, we can show you the overall performance picture using our famous value scatter plots. The performance results come from all seven of the games we tested, averaged via a geometric mean to reduce the impact of outliers. The prices come from current listings at Newegg for the exact cards we tested. As always, the most desirable combinations of price and performance will be located closer to the top left corner of the plot.
Pop back and forth between the 99th percentile and average FPS plots, and you’ll see two different stories being told. The FPS average suggests near-parity performance between the 7950 and the GTX 660 Ti, with a tiny edge to the GeForce. The 99th percentile frame time, though, captures the impact of the Radeon’s frame latency issues and suggests the GTX 660 Ti is easily the superior performer. That fact won’t be a surprise to anyone who’s read this far.
Armed with that info, we can dispense with the talk about game bundles, rebates, and pricing shenanigans that might shift the value math in favor of one camp or another. Instead, we have a crystal clear recommendation of the GeForce GTX 660 Ti over the Radeon HD 7950 for this winter’s crop of blockbuster games. Perhaps AMD will smooth out some of the rough patches in later driver releases, but the games we’ve tested are already on the market—and Nvidia undeniably delivers the better experience in them, overall.
I’m forced to be concise on Twitter.