3DMark03 controversy refuses to die
FutureMark just released a new patch for its 3DMark03 graphics benchmark, and as you might expect, the update has spawned a new controversy. The performance of GeForce FX-based graphics cards suffers under the new patch, which apparently breaks a number of application-specific optimizations.
Parts of the program code have been changed so that possible 3DMark03 specific optimizations in current drivers will not work. 3DMark03 specific optimizations in drivers are against run rules of 3DMark03, because they invalidate the performance measurement results and thus make it impossible to compare performances of different hardware. A list of drivers that have been tested - and confirmed to produce valid 3DMark03 scores - has been published on Futuremark’s website.
FutureMark says that NVIDIA's 52.16 drivers are approved for valid 3DMark03 scores, but at least one of NVIDIA's partners has accused
FutureMark of disabling the driver's unified compiler. A disabled GPU compiler is blamed for the performance hit GeForce FX cards are taking with the new patch, but according to FutureMark's Tero Sarkkinen, that's not technically feasible because 3DMark03 doesn't have direct access to the graphics driver. Here's what Sarkkinen has to say about some more specific build 340 changes:
The only change in build 340 is the order of some instructions in the shaders or the registers they use. This means that new shaders are mathematically equivalent with previous shaders. A GPU compiler should process the old and the new shader code basically with the same performance. Of course, if there are application specific optimizations in the driver that depend on identifying a shader or parts of it, then you might see performance differences because these optimizations will not work if the driver is not able to detect the shader.
NVIDIA has yet to comment on the issue, and it will be interesting to see what their official response is. According to NVIDIA's new driver guidelines, the 52.16s should be devoid of any benchmark-specific optimizations. The 52.16's compiler is free to re-order shader instructions as long as rendering output doesn't change, so the compiler may be hitting a snag with build 340's reordered shader instructions.
For what it's worth, the performance of ATI's Radeons appears to be consistent between build 330 and 340 of 3DMark03.