Wow. There seem to be a lot of angry and/or defensive NVidiot Gerbils in here today.

Yes, NVidia cheated blatantly, and they got caught red-handed. This was not a driver bug or an optimization. It was a cheat done strictly to falsely make the GeForceFX 5800 Ultra's performance look better than it really is.

ATI's change was closer to an optimization, since they substituted shader code that produced the same output as the 3Dmark03 code, but runs slightly faster on the R300 and R350 VPU than the original code does.
http://www.beyond3d.com/#news6040
Chris Evenden explained it this way:
We render the scene exactly as intended by Futuremark, in full-precision floating point. Our shaders are mathematically and functionally identical to Futuremark's and there are no visual artifacts; we simply shuffle instructions to take advantage of our architecture....However, we recognize that these can be used by some people to call into question the legitimacy of benchmark results, and so we are removing them from our driver as soon as is physically possible. We expect them to be gone by the next release of CATALYST.

Buub, there are plenty of equally valid "bottom lines":

"Either you render the same output, or you don't" <-I recommend you look at some screenshot comparisons, and consider that just maybe Futuremark isn't just flat out lying in statements nVidia hasn't dared to contradict yet.

"Either you optimize in a way that can be done in games, or you don't"<-Tim Sweeney's comments might serve to illustrate something along these lines.

These bottom lines differentiate between ATI and nVidia, and you are working hard to ignore them so you can feel good about saying "nVidia is no worse than ATI".

Now, if you want to look at the full picture instead of the little corner that fits into your bias, what it indicates is that ATI's optimization is not a cheat (and nVidia's is), but that this optimization not being applied unless recognizing 3dmark 03's shader file did break Futuremark's rules.

The difference between nVidia and ATI is that a gamer would actually benefit from ATI giving that same special treatment to a game: identical, but faster, output for every usage (i.e., off the rail) of optimized shaders in the game. However, a gamer would be losing image quality from some of nVidia's changes, and seeing the performance benefit from others only in timedemos. Is that difference insignificant to you?

The similarity between what they did is that they broke Futuremark's rules, but to insist that this makes them equivalent is like equating J-walking with armed robbery because they are both law violations. The reason Futuremark has this rule is so that 3dmark only shows optimizations that affect all applications, not just 3dmark 03, and yet at the same you maintain that "2) This proves that synthetic benchmarks need to be taken with a grain of salt, and only considered minor indicators of performance. Real-world tests need to be the primary determining factor", which this rule directly contradicts.

Why do so many people attacking 3dmark stick to arguments ignoring any of the technical rebuttal to nVidia's comments?

Why do people who persist in saying that ATI and nVidia are equally to blame for these issues depend on ignoring the technical analysis of what each did to support their persistence? It is possible to call two different things a "crime" and not then go on and say something preposterous like "all crimes are the same".

I just can't wait for the fools to come out and say "look ati lost some score too so they must have been cheating as well" I'd bet my last $$ it was the result of fixing the benchmarks so that nVidia could not run their cheats anymore...

I wouldn't really call what ATi did a cheat because unlike nVidia, their output was BIT FOR BIT THE SAME as Futuremark's. They simply arranged the pixel shader instructions to be more efficient - BUT THE END RESULTS ARE THE EXACT SAME. That's the nature of optimizations! Making a task more efficient without lowering quality.

Check out Tim Sweeny's post over at beyond3d for more on that. I thought he really made a good point, and as a game developer I give what he says a bit more weight when forming an opinion.

In simple terms its in nvidias best interest to see futuremark die because nvidia is best of the bunch at helping devs optimize thier games to run better and faster on thier stuff.

Make no mistakes this is all about money and the fact that nvidia spends a ton of it to gain real world game performance all of wich is ignored by this benchmark. And nvidia cant have that. Because oems would use the benchmark numbers to pick bargain cards to buy nvidia has to bludgeon the oems with the thought that the benchmark is too flawed to use as such.

And I rather much expect they have managed to bring that thought home.

As for us and nvidia they know we dont trust benchmarks and that we dont trust timedemos and that instead we trust our friends and trusted sites to pick the cards that realy work well.( wich happens to be why they hauled ass getting the nv35 out;/)

I dont think nvidia did anythign in to the benchmark they dont do to any important game.. the only problem being that unlike in a game they couldnt run all the variations through to test for bugs and thus I suspect an automated shader swapper driver dodad swapped in a bit too agressive a shader optimization.

Sound far feched? Well what do you think 10-20000 high powered puters are doing at nvidia day and night? Get a clue they have to be doing one hell of alot. And what would happen is they let those puters chew on that benchmark even for just one day...?

#60: You fail to realize that Nvidia has sub-par image quality with AntiAliasing when compared to ATI. The gamma-corrected sparse-grid AA (ATI solution) really outshines the non-gamma-corrected ordered-grid AA (Nvidia solution).

Some even say that ATI's 2xAA looks as good as Nvidia's 4xAA, while ATI's 4xAA is far ahead of even Nvidia's 8xAA.

--|BRiT|

I wonder if it would be too much for Tech-Report to adopt a more nV News style of benchmarking for their graphics cards comparisons? I'm talking about in-game benchmarks here as with this news it's hard to trust timedemos since they too are so easy to cheat and optimize for while being very hard to detect.

To be perfectly blunt I saw this all before and I already know the outcome.

It happened a long time ago for cpus and now its gpus turn. As the cpu companies created more and more complex cpus with more and more features that were harder and harder to understand they had to go out and help companies optimize programs for thier cpus.

At first people screamed bloody murder at the thought of optimizing benchmarks for each cpu but in the end there was no choice at all. And there is no choice now. Sooner or later the difference between an unoptimized "sythetic" benchmark and real world results will be massive so massive no one can use them anymore save maybe to benchmark such silly things as memory throughput and agp or pci express throughput and theoretical max render rates and so on.

what really pissed me off is that these a**holes (both nvidia & ati) are spending their time coding new ways to cheat on benchmarks instead of make them work better, fix bugs, and go for more general imporvements. The det FX pack weighed in at 12+ megs, while the previous release was 8. How much of that is spurious code that is for cheating on 3dMark? How many real games are slowed down while the F'ing drivers are looking to see whether this app is a benchmark that they have hard-coded optimizations for?

I care a lot more about the time-wasting and bloat-production aspects of this than the whole "ooh they are cheating" thing. There are better uses for a team of skilled coders. Work on the linux drivers maybe!

What is the possibility of something like the following happening? ...

We can reveal that Valve is currently looking at making its massively
anticipated FPS sequel Half-Life 2 exclusive to nVidia graphics cards, in
what will be the first deal of its kind in the PC gaming sector.

This massive story comes from insiders at nVidia, who are boasting that this
will be the first of many such deals for exclusive content, bespoke to its
cards.

We have been aware for some time that both ATi and nVidia have been courting
the publishers and developers of AAA PC games, trying to gain
card-exclusivity.

"You will see a massive change in the structure of the PC games industry in
the next few months, with the first batch of announcements hitting at E3,"
one insider told us. "Instead of being referred to as PC gaming, you will
become an nVidia or ATi gamer, as distinct as Sony and Nintendo."

This concept has been circulating for some time, with gossip about graphics
cards overshadowing next-gen console talk at the recent Game Developer's
Conference.

Remember where you read this first, and expect confirmation at this May's
E3.

Futuremark did state they are investigating the large (~8%) drop in performance by the Radeon 9700/9800's in Game Test 4.

Does this mean Damage will rerun the Detonator tests that were done earlier?

Apparently ATI cheated a tad as well:
"Our investigations reveal that some drivers from ATI also produce a slightly lower total score on the new build of 3DMark03. The drop in performance on the same test system with a Radeon 9800 Pro using the Catayst 3.4 drivers is 1.9%. This performance drop is almost entirely due to 8.2% difference in the game test 4 result, which means that the test was also detected and somehow altered by the ATI drivers. We are currently investigating this further."

Bah. That whitepaper needed another grammar check before being pushed out the door.

"Second, in well-designed benchmarks like 3dmark03, all cards are instructed to do the same amount of work."

.. like using different PS paths? The final result may (should!) be the same, but the amount of work is certainly not. You could argue that using preset clip planes is a performance tweak, not an optimization, in the sense that the final result is the same -- and it might even be true, if it wasn't special-cased for this particular application and thus more generally useful.

It would suck to be a software engineer at nvidia right now. I wonder whose decision it was to implement these driver cheats.

My brief comments:

FM's business model is poor and distracting to the industry. Writing DX9 code seems to be so based on the connection between the developer and the IHV that anything but general scores from 3D Mark represent influence more than anything. I think Nvidia has stepped back and seen this. I don't think FM has any place paving the way or setting standards. I just don't like the flow of money involved. 3D Mark should be benchmarking after the fact, but I think there's a lot more than that going on. The developers should be leading the way, and the IHVs should be luring them with easy to use, powerful, and compatible solutions (which brings us to the whole HLSL discussion).
Politics and money. Bleh.

Nvidia is using methods to increase the framerates of 3D Mark 03 that would just not be practical in many real games. It is for that reason that I would lean towards the "cheat" column. Also, it can't be expected for Nvidia to do such optimizations for all games. At best the methods they use for 3D Mark would make their way to a handful of popular games (Doom 3, HL2 anyone?).

I think Nvidia knows what they're doing. They're picking a fight, and I doubt they're going to lose. FM doesn't have the clout, and in mind they shouldn't. Nvidia is going to draw this back into a battle of HLSL vs. HLSL, where ATI and Nvidia have to battle over developer mindshare, capitalist/darwinian style, strictly where the battle should be taking place. FM shouldn't be in the drivers seat when it comes to PC graphics. I think we'll look back in five years and see synthetic benchmarks as a distraction from a proper goal.

Like it or not, the barrier for entry in PC graphics is just skyrocketing. HLSL is as inevitable as C is over assembly. Bitch and moan all you want, but sit for five years and see what happens.

That's about all I care to say on the issue. I've never really given much of a crap for 3D Mark, anyway. It isn't worth the publicity it gets, and it really isn't worth my time or the hype of any scandal that would ever arise.

Don't buy from the cheating bastards. Now we have Dawn running ATI, so what's the need? ;)

But you couldn't be more wrong, Geoff! nVidia has commented, at least according to Blues News, this is what nVidia has said in response to FutureMark's exposure of its cheating:

"Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in), we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer," the representative said. "We don't know what they did, but it looks like they have intentionally tried to create a scenario that makes our products look bad."

Wow, what a well-reasoned, intelligent reply....*chuckle* (deeply sarcastic)

Poor little nVidia--getting kicked around by the behemoth FutureMark like that. Imagine--nVidia quit the FM beta program and so nVidia has no 'tools' to tell it when it is cheating in 3D Mark and when it is not! Oh, poor thing. Yea, since nVidia quit the FutureMark 12-step-program the company no longer has anyone to HELP IT STOP CHEATING. So how can you blame them for cheating--and not even being aware of of cheating when they cheat--and thinking FutureMark is just "intentionally picking on them".

Oh, brother...now, I think I've heard everything. Siyonara, nVidia--alas I knew him well.

the latest drivers from ati (expanded) weight in at exactly 7.12MB that covers ALL of the Radeon cards from the Original 32MB SDR all the way up to the 9800Pro

Just to be fair ATi has a seperate control panel application that weighs in at about 6.5MB compressed.

DX9 shader 2.0 for the NV35 and NV30 is Pathetic!!! It's less than half of the 9800pro. ATi is future proof when it comes to DX9.

Go Nvidia. I like corporate warfare!

this is finally another source besides ETs that states anything about the "cheat".