AMD is keen to talk about a provocative white paper that it commissioned from the testing firm QA Consultants. In its conclusions, the firm says AMD has the most stable graphics driver in the industry.
How did QA Consultants come to that conclusion? The firm ran a single test, called CRASH, that's part of Microsoft's Hardware Lab Kit (or HLK), a suite of tools designed to help companies ascertain whether their products qualify for the Windows Hardware Compatibility Program. CRASH is just one of a dizzying array of graphics-related tests available from that toolkit.
The Radeon Settings app
While Microsoft doesn't document exactly what tests CRASH performs on a system, AMD says it “contains a variety of graphical functions across DirectX 9, 10 and 11 including changes in resolution, color settings, screen rotations, color overlays, sleeping and waking up.” The test runs for four hours, and Microsoft says “if the system does not stop responding or encounter a bug check during this specified time, the test is considered a pass.”
QA Consultants went one better, perhaps, by running looped instances of CRASH 24 hours a day for 12 days straight on six graphics cards from each vendor: three consumer-grade cards with consumer drivers, and three pro cards with pro drivers.
The firm logged the number of attempts that succeeded versus the number that failed in order to arrive at its ultimate conclusion: AMD cards passed 93% of the 432 test attempts, while Nvidia cards passed 82% of those attempts. Impressive, on the face of it.
Questions about going above and beyond Microsoft's test parameters aside, though, AMD's description of CRASH suggests it doesn't perform any application-level stress testing with popular games or professional software that might allow users to draw conclusions about the relative stability of each vendor's driver for their own use cases.
While CRASH could certainly be “intense,” as AMD puts it, the direction of that intensity is seemingly meant to ensure that Windows 10 itself won't have a fit with a given piece of hardware and its driver stack, not to test the stability of the applications that run atop the operating system. We've asked AMD if the scope of the CRASH test extends beyond the description the company offered in its blog post, and if so, how far.
In short, AMD's consumer and professional graphics drivers might be more stable than Nvidia's on a single test or a narrow range of tests, but users should carefully read the fine print before taking as gospel the superiority of one vendor over another. Unless you're frantically changing display resolutions or color settings or rotating your display between portrait and landscape modes often, these tests might not be relevant to you.
We'd be curious to see a similar stress test performed with a basket of games or 3D applications that gamers and professionals are likely to employ in their day-to-day work or play to put these numbers in richer context.
Nvidia offered no specific comment in response to QA Consultants' results.