Smartphones are becoming more like PCs by the day. Now, they've taken another step toward complete PC-ification by learning to cheat in popular benchmarks. Ain't progress grand?
AnandTech exposed the subterfuge on Samsung's Galaxy S4. The site started by testing the international version of the phone, which is powered by Samsung's own Exynos 5 system-on-a-chip. AnandTech ran a shell command to repeatedly probe the clock speed of the SoC's PowerVR integrated graphics. It found that, while the clock speed never rose past 480MHz in games—even "the most demanding titles"—running benchmarks like GLBenchmark, AnTuTu, and Quadrant bumped the speed to 532MHz.
The same thing happened in CPU tests, only this time, both the Exynos-powered phone and its Qualcomm-powered North American counterpart skewed the numbers. As AnandTech explains, the Qualcomm-powered model "is set to the maximum CPU frequency available at app launch and stays there for the duration, all cores are plugged in as well, regardless of load, as soon as the application starts." AnTuTu, Linpack, Benchmark Pi, and Quadrant all exhibit the same behavior.
The culprit? A set of exceptions in the phone's DVFS (short for dynamic voltage and frequency scaling) management app. Opening the app with a text editor reveals incriminating text strings, including "BenchmarkBooster" as well as specific references to the aforementioned benchmark apps.
In a statement sent to AnandTech this morning, Samsung claims the Galaxy S4's graphics speed maxes out in "apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance." It goes on to indulge in a bit of doublethink, adding, "The maximum GPU frequencies . . . were not intended to improve certain benchmark results." AnandTech found that the graphics speed did indeed reach its peak in the Camera app, but only for short bursts. In benchmarks, the peak speed was sustained.
When I went hands-on with the Galaxy S4 last month, I noticed that the device felt slower than my iPhone 5, even though benchmarks around the web suggested the opposite. Perhaps this discovery helps explain that. In any case, Samsung needs to learn the same lesson its PC industry counterparts did some years ago: funny business in benchmarks always comes to light sooner or later, and it's never a good alternative to winning performance contests the right way.