AMD touts driver stability on the back of new test report

AMD is keen to talk about a provocative white paper that it commissioned from the testing firm QA Consultants. In its conclusions, the firm says AMD has the most stable graphics driver in the industry.

How did QA Consultants come to that conclusion? The firm ran a single test, called CRASH, that's part of Microsoft's Hardware Lab Kit (or HLK), a suite of tools designed to help companies ascertain whether their products qualify for the Windows Hardware Compatibility Program. CRASH is just one of a dizzying array of graphics-related tests available from that toolkit.

The Radeon Settings app

While Microsoft doesn't document exactly what tests CRASH performs on a system, AMD says it “contains a variety of graphical functions across DirectX 9, 10 and 11 including changes in resolution, color settings, screen rotations, color overlays, sleeping and waking up.” The test runs for four hours, and Microsoft says “if the system does not stop responding or encounter a bug check during this specified time, the test is considered a pass.”

QA Consultants went one better, perhaps, by running looped instances of CRASH 24 hours a day for 12 days straight on six graphics cards from each vendor: three consumer-grade cards with consumer drivers, and three pro cards with pro drivers.

The firm logged the number of attempts that succeeded versus the number that failed in order to arrive at its ultimate conclusion: AMD cards passed 93% of the 432 test attempts, while Nvidia cards passed 82% of those attempts. Impressive, on the face of it.

Questions about going above and beyond Microsoft's test parameters aside, though, AMD's description of CRASH suggests it doesn't perform any application-level stress testing with popular games or professional software that might allow users to draw conclusions about the relative stability of each vendor's driver for their own use cases.

While CRASH could certainly be “intense,” as AMD puts it, the direction of that intensity is seemingly meant to ensure that Windows 10 itself won't have a fit with a given piece of hardware and its driver stack, not to test the stability of the applications that run atop the operating system. We've asked AMD if the scope of the CRASH test extends beyond the description the company offered in its blog post, and if so, how far.

In short, AMD's consumer and professional graphics drivers might be more stable than Nvidia's on a single test or a narrow range of tests, but users should carefully read the fine print before taking as gospel the superiority of one vendor over another. Unless you're frantically changing display resolutions or color settings or rotating your display between portrait and landscape modes often, these tests might not be relevant to you.

We'd be curious to see a similar stress test performed with a basket of games or 3D applications that gamers and professionals are likely to employ in their day-to-day work or play to put these numbers in richer context.

Nvidia offered no specific comment in response to QA Consultants' results.

Comments closed
    • Krogoth
    • 1 year ago

    Both camps are about the same. They each have own set of teething to stupid issues which the fanboys blow out of proportion.

    • techguy
    • 1 year ago

    I don’t know why anyone is discussing the particulars of this study, it was paid for by AMD. That completely invalidates it.

    New tobacco company study says tobacco good for you, nicotine patches and gum cause cancer.

      • swaaye
      • 1 year ago

      Excuse to talk about one’s suffering and start the complaining!!!

    • DPete27
    • 1 year ago

    When’s the last AMD driver that was WHQL certified?… if AMD is sooooo good at it.

      • Kelbor
      • 1 year ago

      DPete27, you have to remember that it takes extra time and is expensive to submit a driver to Microsoft for a WHQL certification. It’s just not cost effective to certify every driver update. The last WHQL driver was on February 7th 2018. AMD has put considerable money and talent in redeveloping their drivers and have improved greatly from it.

    • DoomGuy64
    • 1 year ago

    This should have been common knowledge ever since Vista, when Microsoft complained about Nvidia’s stability.

    Not to say AMD doesn’t lack some functionality compared to Nvidia, but they’ve caught up since crimson. I will say however, that while desktop productivity is stable, AMD’s control panel is often not. Not that they don’t fix most real issues, but their game profile section can lag and crash even with an SSD, and that’s NOT A BUG, but how the profile tab actually functions when a lot of games are installed. That said, I don’t know if the latest driver fixes that, since I haven’t installed it, but IMO the profile tab is badly designed with no search function, and defaults to loading all profiles on the first click, which is ludicrous. Global should be first, profiles second.

      • DPete27
      • 1 year ago

      Not sure what you’re getting at. Global is global. All games get a profile tab, but you have to manually enable them in order for a given profile to override the global settings.

      I’ve had more problems with Global WattMan forgetting my undervolts, while Profile WattMan settings stick properly.

        • DoomGuy64
        • 1 year ago

        How do you not get it? Clicking the gaming tab loads all profiles by default (unchangeable), and I have a steam achievement for owning 1k games. Everything I have installed is profiled, and it lags the control panel like hell to open the gaming tab.

        You CAN’T get into global, without first loading every single detected game on your system, and there is no search function to instantly go to a certain game if I wanted. Games like Deadspace 3 can’t handle forced AA, so it is useful occasionally. It doesn’t matter if I’m enabling a profile or not, because I mostly use global settings, but those game profiles are loaded whether or not I use them, and it lags the control panel. Clicking out of the tab before the profiles are fully loaded also crashes the control panel. IMO, profiles should be optional like before in CCC, and not mandatory as now.

        And yes, global wattman has issues too, and I don’t know if AMD is ever planning on fixing that, because I’ve heard that was considered a safety feature, and I more or less load a saved profile when it forgets, because it’s a hassle to profile every game. It’s probably better to use MSI afterburner for this stuff anyway, considering AMD’s track record of buggy overclocking support.

    • ronch
    • 1 year ago

    I’m actually still running 2014 drivers with my HD7770. Newer drivers slowed my boot up times and I’m not really gonna miss Crimson. I’m not even sure if it’s WHQL certified but it runs just fine. Some textures flicker in Unreal Tournament (the original) but so what.

    • Unknown-Error
    • 1 year ago

    AMD and Drivers are like…………………. ¯\_(ツ)_/¯

    • reignofchaos
    • 1 year ago

    The actual report is here:

    [url<]https://www.amd.com/system/files/documents/graphics-driver-quality.pdf[/url<] Here are the individual GPUs and number of times they failed: Radeon RX 560: Failed 1/364 times Radeon RX 580: Failed 2/364 times Radeon RX Vega 64: Failed 2/364 times Radeon Pro WX3100: Failed 13/364 times Radeon Pro WX7100: Failed 5/364 times Radeon Pro WX9100: Failed 9/364 times GeForce 1050: Failed 3/364 times GeForce 1060: Failed 10/364 times Geforce 1080ti: Failed 2/364 times Quadro P600: Failed 28/364 times Quadro P4000: Failed 12/364 times Quadro P5000: Failed 18/364 times All I can say from the above is that the so called better stability of Pro cards is complete bull. Buy consumer cards and stay on a stable driver for the most stable experience. Personally I can't deduce anything more from this survey!

      • chuckula
      • 1 year ago

      [quote<]All I can say from the above is that the so called better stability of Pro cards is complete bull.[/quote<] That's not necessarily true at all. Pro cards are supposed to be certified to be stable for certain enumerated applications. Think things like 3D-CAD applications where correctness of the display is far more important than a twitchy FPS count. Nobody ever claimed that the "CRASH" benchmark was supposed to be certified for pro cards, just like how if [insert game of the week here] crashes on a pro card that in no way means the drivers for the pro card are "broken" since that game isn't the intended use for the pro card.

        • reignofchaos
        • 1 year ago

        I’m not referring to certification for pro applications. Those certifications only say the card’s display output will be correct when compared to a reference output which is necessary for especially for pixel correct engineering applications. Some effects or geometry may not even render correctly with consumer cards. If you use such apps, sure use the pro cards.

        I’m talking about general stability which is touted to be better on Quadro or Radeon Pro cards due to better quality and what not which is used to fatten up their prices. That part is complete bull from the above tests.

        If this “CRASH” application runs basic DX9/DX11/GL tests, then the pro and consumer cards should run them without trouble. The pro cards must support every rendering feature what the consumer cards do. Just that they may not run them as fast as the consumer cards. However they are also fully compatible with DX11/DX9/GL.

      • stefem
      • 1 year ago

      The problem is that the whole testing is bullshit, it was commissioned and paid by AMD, the testing itself is hardly representative of any real world scenario and the OS and driver selection is odd. Win1803 was a bug-riddled mess, and they didn’t use NVIDIA’s 1803 drivers for Quadro (R396), which were out at the time.

    • ColeLT1
    • 1 year ago

    Willing to bet they didn’t test Vega56 cards, most the unstable cards I have ever owned. They would “unload” in under 24hrs of PC powered on, had to go to device manager and disable/re-enable daily. Maybe the drivers have improved, but I had to ebay them quickly. Issue free with 1080ti.

    • TwistedKestrel
    • 1 year ago

    On the one hand, I’ve been mostly happy with AMD drivers for the past … years (4670 came out when again?)

    On the other hand I immediately had flashbacks to the critical Overwatch and Titanfall 2 bugs that were around for MONTHS

    • synthtel2
    • 1 year ago

    That’s all they bothered doing to them, and both still failed that many tests? I’ll stick to assuming all graphics drivers are garbage and being pleasantly surprised when they turn out to be otherwise, thanks.

    Really though, there are plenty of bugs to go around. I find the AMD side of that more workable, but that doesn’t mean it’s good in any absolute terms.

      • chuckula
      • 1 year ago

      From what I can gather of this “CRASH” test here’s an easy explanation for their convoluted numbers:

      1. Both the AMD and Nvidia drivers have a memory leak that causes these failures.
      2. AMD’s memory leak is slower, hence the smaller number of failures over the testing period.

        • MOSFET
        • 1 year ago

        In that case (memory leak), Intel graphics drivers would be sitting on a goose egg success rate.

      • Action.de.Parsnip
      • 1 year ago

      That’s the really amusing part it’s just spamming little things and they still fall over, which shocked me a little to be honest. Gut feeling is its the sleep/wake cycles hardware on/hardware off while maintaining system state and all the many little hoops that entails jumping through.

    • not@home
    • 1 year ago

    Well my 7870 will not wake up the monitors if they go to sleep, so I have to turn off the monitors manually and disable sleep mode. Also, I cannot play Ark – Survival Evolved because after about 5-10 minutes my whole PC crashes. I would say AMD’s drivers are not stable enough.

      • kuttan
      • 1 year ago

      Get a quality power supply than to blame on drivers.

        • not@home
        • 1 year ago

        The whole rest of the PC is only about a year old now, Including a Seasonic 750w PSU. I do not play many games, so I do not have a large sample size, but Ark is the only game I have problems with. The issue with the monitors not waking back up is definitely a driver issue. AMD fixed the issue for one driver release about two years ago, when I had the GPU in my old PC before it died of unrelated issues, But the next driver release had the bug back in it. It is a known issue that AMD does not care enough about.

        Edit: [url<]https://community.amd.com/thread/199688?q=sleep[/url<]

          • Action.de.Parsnip
          • 1 year ago

          Are you sure *sure* you don’t have any other system gremlins? Oddball issues can sometimes be an undiscovered side issue. Yearssss ago I had a soundcard (yeah i know) that went bad and started throwing all manner of weird issues.

            • freebird
            • 1 year ago

            yeah, a couple of weeks ago my PC stopped waking up (turning the screen back on) whenever the screen went blank. Thought it was a Windows update or something, because I didn’t update much else nor my GPU drivers (2 Vega56s) but found the root cause/program was my XMR mining software (which didn’t get updated). So I switched back to mining ETH with different software. No more wakeup problems.

            I think it was some windows update and interaction with the XMR mining software, but who knows. I could’ve blamed the GPU drivers, but everything is fine now with the same GPU drivers.

      • K-L-Waster
      • 1 year ago

      That’s not sounding like drivers, that’s sounding like one component or another is getting ready for involuntary retirement…

      • BurntMyBacon
      • 1 year ago

      They tested 6 graphics cards from each company and only 3 of them were consumer grade. I’m thinking the chances of you 7870 being well represented here are a bit slim.

      On the other hand, the problems you mention sound less like driver issues than hardware issues. Can’t say without further information and testing, but you may want to look into it if only to rule it out.

      • anubis44
      • 1 year ago

      And get a similar problem with a GTX1050, but not either one of my Radeon cards on my HTCP. They can wake up my 42″ RCA 1080p TV, but not the nVidia card, using display port. So I’m using an R9 290 in my HTPC, which is certainly much more powerful than the 1050 had originally bought for it. AMD cards/drivers NEVER crash on either my wife’s PC, or my HTPC. The nVidia driver has crashed, albeit quite infrequently, on my main rig with that 1050 card in it. Waiting for the next compelling AMD Radeon card to replace that 1050.

        • chuckula
        • 1 year ago

        Yeah, you used an Nvidia product.

        Caught the lie early on that cut-n-paste job there.

        • DoomGuy64
        • 1 year ago

        The Hawaii chips seem to be the most driver stable of AMD’s products, although Polaris doesn’t seem to have a bad rep either. I had issues with Fury on my freesync monitor, so I sold it and went back to my old 390, which still can play games decently enough @ 1440p.

      • Jigar
      • 1 year ago

      That doesn’t sound like driver at all. You might want to recheck other components.

    • Redocbew
    • 1 year ago

    I have this nostalgic memory of “white papers” being set apart from ordinary documentation. They weren’t written for the average user, and that was fine because you didn’t read the whitepaper on something just to figure out how to use it. You read the whitepaper to figure out why it worked, and to jump down the rabbit hole to figure out exactly what was going on in there even at the lowest level.

    Perhaps that’s a romanticized memory and the reality was more complicated, but this is a far cry from what I knew white papers to be and what got me interested in technology in the first place.

    Oh, and to anyone who thinks I’m just throwing mud on AMD for the fun of it and wants to jump on the bandwagon: I hate you also.

      • derFunkenstein
      • 1 year ago

      well the feeling is not mutual. [url=https://www.youtube.com/watch?v=ZGQyv-43334<]I love you, man![/url<]

    • rnalsation
    • 1 year ago

    This still won’t make me forget AMD breaking support for many DX9 titles a few drivers ago. Or only releasing 1 somewhat buggy driver for many of their APUs under Windows 10 and forgetting about them. Or their final driver release for many products (before their new UI) having a broken HDMI audio driver.

    Edit: I’m not saying I’m an NVIDIA fan, but AMD has burned me too many times on their GPUs for me to do them again. Matrox forever!

      • Concupiscence
      • 1 year ago

      The only unstable experience I had with AMD was flogging OpenCL for hours on end with BOINC, and more specifically, Einstein@Home. Beyond that the Nvidia and AMD experiences have been indistinguishable for me for years.

      As for Matrox, the Parhelia-derived M9120 I grabbed a while ago on a lark is stable. The 3D is unutterably pokey – we’re talking maybe as fast as a Geforce 6200 – and anything involving GLSL is probably going to be buggy as all get out, but it DOES work. Their newer solutions are all AMD-based and Matrox’s reputation for software is not especially stellar, but they are still in business…

        • rnalsation
        • 1 year ago

        To be fair, I haven’t used a Matrox card since the 90s. The ATI 9600 did treat me well for years until the driver abomination that was the HD 2600 got into my new Vista System.

          • anubis44
          • 1 year ago

          Speaking of horrible drivers in Windows Vista, nVidia actually took the pole position on that: [url<]https://www.youtube.com/watch?v=dE-YM_3YBm0&t=2s[/url<]

            • rnalsation
            • 1 year ago

            The tone of that video really ruins any factual information to be gained from it.

      • Jigar
      • 1 year ago

      The only reason i went back to AMD and never came back to Nvidia is because Nvidia’s one famous driver killed my 8800 GT. Never touched Nvidia ever since.

        • rnalsation
        • 1 year ago

        The 8800s (and others) would die on their own w/o driver intervention. They had a manufacturing defect that was exasperated by heat.

          • Jigar
          • 1 year ago

          My Card was modded with Intel Cooler. You can google for pics. It had amazing cooling temps never went above 55 C.Yes Intel stock cooler was installed and it worked amazing for 8800GT.

            • K-L-Waster
            • 1 year ago

            Mods card, then when card dies blames drivers.

    • chuckula
    • 1 year ago

    Oh yeah, figured it out. Page 3 of the whitepaper clearly lists:
    [code<]Intel® Core™ i5 8400 CPU MSI Z370-A PRO Motherboard ADATA XPG Z1 16 GB DDR4 RAM Corsair CX Series CX750 PSU Samsung 860 EVO 250GB SSD[/code<] It's a well-established fact that Nvidia conspires with Intel to illegally gimp AMD products, which has the unintended side effect of making them more stable since Intel's CPUs are hard-wired to prevent AMD GPUs from hitting 100% performance. I'd bet that a superior ThreadRipper2 (TM) gaming platform would clear these numbers right up.

      • Mr Bill
      • 1 year ago

      what’s sauce for the goose is sauce for the gander

        • chuckula
        • 1 year ago

        Are you suggesting that Intel [b<]ALSO[/b<] colludes with AMD to illegally gimp the performance of AMD products when installed on AMD products?!?!?!? Because if so, then I would like to subscribe to your newsletter.

      • kuttan
      • 1 year ago

      Your desperation to troll “AMD Hate” on almost every article just stinks. The funny thing is AMD is getting stronger after each day. Get a life Jokeckula!

        • chuckula
        • 1 year ago

        [quote<] The funny thing is AMD is getting stronger after each day.[/quote<] Good, then their graphics division better show strong double-digit sales growth of the same old products [b<]AFTER[/b<] the mining crash is taken into account. Funny how Nvidia manages to post real profits while not even having the cash-cow x86 license to fall back on and not really even selling products in the mobile space either.

          • anubis44
          • 1 year ago

          There’s nothing funny about how nVidia manages to make ‘real profits’. They like to cheat, gimp their products and market them as higher performing than they are, and rip off their own customers. No mystery there at all.

            • chuckula
            • 1 year ago

            Your desperation to troll “Nvidia Hate” on literally every article just stinks. The funny thing is Nividia is getting stronger after each day. Get a life Kuttan/Anubis/Sockpupet accounts!!

      • Jigar
      • 1 year ago

      WCCF.com would love you for commenting on their news threads. Or are you already there ?

      • Amiga500+
      • 1 year ago

      Chucky, there is a fine line between being the funny cool kid and trying too hard….

    • chuckula
    • 1 year ago

    [quote<]AMD is keen to talk about a provocative white paper [u<][b<][i<]that it commissioned[/u<][/b<][/i<] from the testing firm QA Consultants. [/quote<] Emphasis. Added. In other news, QA Consultants. at least meets the honest politician standard: Once bought it stays bought.

      • Fonbu
      • 1 year ago

      I would imagine… Usually a report like this has some bias within, half truths and only tells part of the story.

      • Amiga500+
      • 1 year ago

      They likely ran a whole rake of tests using various things within the HLK, and this was the only one that clearly differentiated AMD as being arguably “better”.

Pin It on Pinterest

Share This