BAPCo, the non-profit consortium responsible for the SYSmark benchmark, appears to have lost a number of rather important members recently. SemiAccurate broke the story yesterday, reporting that AMD, Nvidia, and Via have all quit the group over what was included in SYSmark 2012.
While it appears that AMD was successful in getting a number of real-world workloads included in the SYSmark suite, the weighting of the results seems to be a major point of contention. Of the 390 variables SYSmark measures across 18 applications, Dessau says the overall score is predominantly defined by less than 10% of the variables tested across just seven applications. He goes on to say that there’s too much emphasis on optical character recognition and file compression. As one might expect, Dessau also takes issue with the absence of workloads that tap the parallel processing power of modern GPUs. Everyday applications that make use of GPU resources may be relatively few and far between, but how often are you waiting on an OCR routine to make sense of a scanned document?
AMD and other BAPCo members that might want to make changes to the benchmark are supposed to be able to offer their proposed modifications for a vote. However, SemiAccurate asserts that Intel "owns the process, and overrides anyone’s views, thoughts and additions." The other BAPCo members listed on the consortium’s about page include Dell, HP, Hitachi, Lenovo, Microsoft, Samsung, Seagate, Sony, and Toshiba. None of them make CPUs—or GPUs, for that matter.
Interestingly, BAPCo’s about page doesn’t list AMD, Nvidia, or Via as members. Google’s cache reveals Via was on the member list as of last week, giving credence to the report that it has also dropped out of the group.