Several Android smartphone makers have been busted for artificially inflating benchmark scores. The roots of this story can be traced all the way back to June, when a member of the Beyond3D forums discovered benchmark-specific optimizations in the Samsung Galaxy S4. AnandTech's subsequent investigation revealed that the device's power management allowed higher GPU frequencies when certain benchmark applications were run. The site also discovered that launching certain benchmarks would cause all CPU cores to become active and run at their highest possible frequency.
Ars Technica has since detailed similar behavior in the Galaxy Note 3, and AnandTech has published a damning report exposing benchmark-related tweaks in not only devices from Samsung, but also ones from Asus, HTC, and LG. Of the handsets tested by AnandTech, only the Motorola products and Google's Nexus devices were devoid of benchmark-specific optimizations.
The optimizations seem to be limited to popular synthetic tests like 3DMark, AnTuTu, and Vellamo. FutureMark expressly forbids optimizations, noting that "drivers may not detect the launch of the benchmark executable and alter, replace or override any parameters or parts of the test based on the detection," but that hasn't stopped HTC and Samsung from gaming 3DMark.
Perhaps the most infuriating thing about this whole story is that Samsung is claiming innocence. The company told CNet that the Galaxy Note 3 "maximises its CPU/GPU frequencies when running features that demand substantial performance," but said "this was not an attempt to exaggerate particular benchmarking results." Ars showed that the Galaxy Note 3 produces much lower Geekbench scores when that software is run with a different name, which seems to contradict Samsung's statement. The optimizations apparently key in on the name of the software rather than the nature of the workload.
We've seen these kinds of shenanigans before, of course. AMD was caught cheating in Quake years ago, and Nvidia gamed 3DMark03. Motherboards sometimes inflate CPU speeds to improve performance, too, although those optimizations aren't benchmark-specific.
It's troubling that shady optimizations appear to be so pervasive in the smartphone space—and that a big name like Samsung is denying any wrongdoing despite pretty convincing evidence to the contrary.
|G.Skill KM560 MX keyboard drops the numpad||10|
|Rumor: Acer Triton 700 may use an unreleased Pascal GPU||27|
|Silverstone Vital VT02 could hold a Core i7 in under two liters||10|
|Galax and KFA2 induct the GTX 1080 Ti into the Hall of Fame||22|
|Acer's Aspire GX-281 lineup brings Ryzen to the masses||18|
|Deals of the week: discounts on CPUs, mobos, and more||10|
|Asetek gets $600,000 from Cooler Master in AIO cooler patent spat||20|
|Acer Predator Triton and Helios laptops are ready for serious play||16|
|Intel enjoys healthy revenue and profits for Q1 2017||31|
|Unless Intel suddenly becomes very aggressive in its pricing, a Skylake-X will certainly cost a hell of a lot more than Ryzen CPU. And who cares if AM...||+66|