BAPCo slams AMD; Nvidia’s departure confirmed

Clearly, BAPCo isn’t too happy with AMD’s decision to pull out of the consortium and disparage the new SYSmark 2012 benchmark. The consortium has issued a stinging statement that explains its side of the story and slams AMD for yesterday’s announcement:

Advanced Micro Devices (AMD) was, until recently, a long standing member of BAPCo. We welcomed AMD’s full participation in the two year development cycle of SYSmark 2012, AMD’s leadership role in creating the development process that BAPCo uses today and in providing expert resources for developing the workload contents. Each member in BAPCo gets one vote on any proposals made by member companies. AMD voted in support of over 80% of the SYSmark 2012 development milestones, and were supported by BAPCo in 100% of the SYSmark 2012 proposals they put forward to the consortium.
BAPCo also notes for the record that, contrary to the false assertion by AMD, BAPCo never threatened AMD with expulsion from the consortium, despite previous violations of its obligations to BAPCo under the consortium member agreement.

BAPCo is disappointed that a former member of the consortium has chosen once more to violate the confidentiality agreement they signed, in an attempt to dissuade customers from using SYSmark to assess the performance of their systems. BAPCo believes the performance measured in each of the six scenarios in SYSmark 2012, which is based on the research of its membership, fairly reflects the performance that users will see when fully utilizing the included applications.

Of course, while BAPCo points the finger at AMD in particular, the chipmaker isn’t alone. SemiAccurate broke the news two days ago that Nvidia and Via have also pulled out, and AnandTech now says it received confirmation from Nvidia that the firm has indeed left BAPCo. "No reason was given," AnandTech adds. So far, though, AMD has been the only one to come out publicly against SYSMark 2012.

Comments closed
    • NIKOLAS
    • 8 years ago

    If you don’t have a CPU worth a damn, then obviously SYSmark 2012 isn’t going to do you any favours.

    • Deanjo
    • 8 years ago

    [quote<] AMD voted in support of over 80% of the SYSmark 2012 development milestones and were supported by BAPCo in 100% of the SYSmark 2012 proposals they put forward to the consortium[/quote<] In other words, they objected to 20% of the proposed tests for a reason but everything that AMD proposed had validity. [quote<]BAPCo is disappointed that a former member of the consortium has chosen once more to violate the confidentiality agreement they signed, in an attempt to dissuade customers from using SYSmark to assess the performance of their systems.[/quote<] If a benching practice is flawed and skews results it should be revealed. There is no valid reason for hiding flawed benching practices other then to skew the results in one vendors favor. Benchmarking without full disclosure is absolutely useless.

    • Spotpuff
    • 8 years ago

    Hopefully the start of the end for benchmark-only apps.

    • Farting Bob
    • 8 years ago

    I never knew benchmarks were so political. Personally i skip the sysmark results in reviews and head to the real world tests. They vary from site to site, but its more realistic of actual performance.

      • wobbles-grogan
      • 8 years ago

      It looks to me like SYSMark are acting like a spoilt child at the moment. All their customers are leaving (which must tell us something about SYSMark, not their customers) and their moaning about it.

      And yes, i generally would look too deeply at the SYSMark benchmarks, and go for the real world ones myself…

        • evilpaul
        • 8 years ago

        I usually skip the “Synthetic Tests” page in reviews as well.

          • khands
          • 8 years ago

          Agreed, I have no use for them.

          • willmore
          • 8 years ago

          I like the true synthetic tests, but I don’t care for the amalgamated suites.

          True synthetic benchmarks are often good at exercising one specific part of the device under test. For example, Stream is good as it tends to show min(memoryBW, FPU thruput). For people doing a lot of vector math, that’s pretty much a basic kernel and can be correlated with real code performance.

          Office suite type of stuff that does a little of this and a little of that and waits for A before doing B rarely correlates well with anything but itself.

            • DarkUltra
            • 8 years ago

            Synthetic tests can show how new hardware architecture and features perform. No games support all new features of a recently a graphics card, or push every feature to its limits. We can see how it would perform in future games that use more advanced shaders or more geometry. That way you only have to lower the AA or resolution, or any other fillrate dependent aspect and still get good performance due to good geometry hardware. But take the results with a grain of salt, and question its neutrality.

      • A_Pickle
      • 8 years ago

      Agreed. I use computers to [i<]actually do things[/i<], like compose written documents, and to create spreadsheets, to surf the web, to play games, to analyze circuits, or a million other things. I care about how those applications perform, and that performance is a metric that can be empirically measured. Given that, why on Earth would I give two stones about synthetic benchmark tests? [quote<]They vary from site to site, but its more realistic of actual performance.[/quote<] But not even by very much -- that's why real-world benchmarks rock. It's the performance numbers that a site provides which one can use to determine the authenticity of the information coming from a given website. There are dozens of PC component review sites out there, and they often test similar programs and they always test similar hardware. The numbers from one site [i<]can't[/i<] be terribly out-of-whack from the rest because that would mean [i<]someone is lying[/i<]. Most of the performance differences you see from site to site are negligible, and can be explained due to minute differences in test configurations, driver and OS versions, etc.

      • UberGerbil
      • 8 years ago

      Benchmarks have [i<]always[/i<] been political. There's too much money, and too many egos, involved for them not to be.

      • AlvinTheNerd
      • 8 years ago

      They are not very important for the enthusiast. I have never seen an ad for an overclocked GPU that stated “+5% in sysbench”

      They are also not very important for servers who have to run a set number of apps very well.

      But they are a MAJOR factor for which computer is purchased for cubicle jungles. IT wants a limited number of models out on the floor to minimize how many different setups they have to deal with and can maximize how many parts can transfer to other computers. Because their are a huge number of applications the cubicle jungle run, many businesses use sysmark and the like to determine which model to buy.

      Most private computers are now laptops. Desktop’s biggest market, by far, is cubicle jungles. And most OEM desktops are built for this environment as well as to maximize sysmark scores. Since sysmark doesn’t give a lot of score increase to GPU’s, most OEM desktops have crappy GPU’s. Since sysmark gives a lot of points for ram increases per dollar, these things have way more ram than they really need.

      Both AMD and Nvidia are pushing computation beyond the CPU. But if cubicle farms are not going to have anything that can use something besides the CPU, they are going to have a hard time getting businesses to build software for these platforms. Its a bad cycle where the business builds software for its current hardware and then buys hardware that maximizes current software. Having sysmark and the like say “There is a huge performance advantage to the GPU and putting some investment in it now will pay off” is the best way to get businesses to start looking at the GPU for more than servers.

      AMD’s and Nvidia’s future is based on that model. Intel’s is based on the CPU. Sysmark 2012 is saying Intel is right.

Pin It on Pinterest

Share This