AMD’s Brazos platform: The benchmarks

Exactly two weeks have passed since I got to sit down at AMD’s Austin campus and benchmark a Zacate development system. You might have read about my experience, which I seasoned with some fresh insights on the Brazos platform that Zacate powers, in my first preview article on November 8.

As I explained then, AMD put a momentary moratorium on the publication of Zacate benchmarks, so the best I could offer were vague and possibly misleading comparisons… and even vaguer hints about the performance of AMD’s first accelerated processing unit (APU). Well, the moratorium finally lifted tonight. I guess the headline was probably a dead giveaway, but still; we’re now free to publish all of the numbers I collected in my few, brief hours of testing.

I’m not going to keep you folks waiting too long for the data—a week is a long enough time to wait. So, we’re going to skip architecture and platform talk. We already provided that information in spades in the aforementioned preview article, as well as Scott’s architectural overview of Bobcat. I should, however, take some time to explain the conditions of the test and lay out a few caveats, since this is one of the very rare cases where we tested neither a retail-boxed computer nor a system we had carefully configured ourselves.

Testing conditions
The misshapen contraption you see above is one of the Zacate test rigs AMD laid out for us lucky testers. Scott saw the same test chassis in San Francisco two months ago, when AMD gave an early demo of its APU running City of Heroes and Internet Explorer 9 with hardware acceleration enabled. This time, however, AMD let me use the system mostly unsupervised for several hours—enough time to run our new mobile benchmark suite, paving the way for a comparison between the Zacate system and several laptops we’ve reviewed so far, including the Nile-powered Toshiba Satellite T235D and a CULV 2010-based ultraportable.

AMD took care of configuring the test machine, outfitting it with the necessary cooling, peripherals, storage, display (an 11.6″, 1366×768 panel), and software. Windows 7 Professional x64 was pre-installed along with a full suite of drivers and some benchmarks:

In a shocking display of rudeness toward my hosts, I pooh-pooed the included benchmarks and instead whipped out a 32GB USB thumb drive containing our mobile test suite—plus a few Steam game backups. Re-installing Windows would have been a good way to ensure a clean testing environment, but that wasn’t really feasible. First, the drivers AMD had installed were pre-release versions not publicly available. And second, I had just enough time with the system to complete my testing. (To give you a rough idea, AMD handed out the keys to the preview system at around 10:00 AM, and I had to head to the airport at around 4:30 PM.)

By the way, that tight schedule also ruled out any battery life tests. Considering AMD quotes run times of at least 8.5 hours for Zacate systems, you can probably guess why.

As far as I could see, though, the Zacate test rig wasn’t up to anything unusual. CPU-Z reported a 1.6GHz CPU clock speed, signaling the quickest Zacate part, and Windows 7 said it had 4GB of memory at its disposal, minus around 400MB requisitioned by the integrated GPU. AMD told us it had set up the machines with solid-state drives, and the Windows Device Manager supported this claim, announcing a 128GB Crucial RealSSD C300. That’s not exactly the kind of drive you’d find on a cheap ultraportable, of course, but its fast read and write speeds proved helpful when installing benchmarks and restoring Steam backups—the clock was ticking, after all.

To sate my paranoid side, I also peeked into the Windows Task Manager and checked the process names. I encountered one service I hadn’t seen before, but after discussing it with AMD, I’m now reasonably satisfied that no tomfoolery was afoot. Eventually, all the poking and prodding led way to some actual testing…

Our testing methods
Over the next few pages, you’ll see the Zacate test build compared to two full-sized Optimus notebooks, the Asus N82Jv and 833Jc; three consumer ultraportables, the Acer Aspire 1810TZ, Acer Aspire 1830TZ, and Toshiba Satellite T235D; one netbook, the Eee PC 1015PN; and one Consumer Ultra-Low-Voltage nettop, the Zotac Zbox HD-ND22.

The 1015PN, N82Jv, U33Jc, and T235D were all tested twice: once plugged in and once unplugged using included “battery-saving” profiles. In the case of the 1015PN, N82Jv, and U33Jc, we compared the battery-saving results to those obtained using built-in “high-performance” modes. The N82Jv’s battery-saving results were obtained with Asus’ Super Hybrid Engine enabled, as well, which dropped the CPU clock speed from 2.4GHz to 0.9-1GHz depending on load. On the U33Jc, the high-performance profile included by Asus raised the maximum CPU clock speed from 2.4 to 2.57GHz. On the Eee PC 1015PN, the included Super Hybrid Engine “Super Performance Mode” raised the CPU speed by 25MHz, while the “Power Saving Mode” limited the CPU to about 1GHz and disabled the Nvidia GPU.

All tests were run at least three times, and we reported the median of those runs.

I should note that, in my hurry, I seem to have misplaced the version numbers for the Zacate system’s graphics and audio drivers. Apologies for that. Also, for what it’s worth, CPU-Z wouldn’t report memory timings.

We used the following versions of our test applications:

• Firefox 3.6.9
• Adobe Flash 10.1.82.76
• x264 HD Benchmark 3.19
• 7-Zip 4.65 x64
• SunSpider JavaScript Benchmark 0.9
• TrueCrypt 7.0a
• Call of Duty 4: Modern Warfare 1.7
• Far Cry 2 1.03

All the tests and methods we employed are publicly available and reproducible. If you have questions about our methods, hit our forums to talk with us about them.

Application performance

SunSpider JavaScript benchmark
The SunSpider JavaScript benchmark has become a staple of browser testing around the web, usually serving to highlight differences in JavaScript execution speeds between browser revisions. Today, we’ll be looking at SunSpider performance with the same browser (Firefox 3.6.9) across multiple notebooks.

Zacate is off to a respectable start, trailing our CULV-based Zbox HD-ND22 by less than 10% and handily outpacing the Eee PC 1015PN’s dual-core Atom running at 1.53GHz.

7-Zip
Odds are, anyone with a computer will need to compress or decompress some files every one in a while. To see how these systems handled that task, we ran 7-Zip’s built-in benchmark and jotted down the results for both compression and decompression.

AMD’s new APU kicks things up a notch in 7-Zip, zooming ahead of entry-level CULV and CULV 2010 systems alike and nipping at the heels of the Toshiba notebook’s Turion II Neo.

TrueCrypt
Next up: file encryption. Because who wants any two-bit thief to have access to his sensitive data? We ran TrueCrypt’s built-in benchmark and averaged the results for all of the different encryption schemes.

x264 video encoding
Last, but not least, we took our notebooks through the x264 high-definition video encoding benchmark.

In TrueCrypt, the Zacate system lies smack-dab in CULV territory. Moving on to a more intense video-encoding test, however, the CULV systems (and even our AMD Nile laptop) speed ahead, leaving the APU in the no-man’s-land between the Atom and the rest of the market. To be fair, video encoders are among those apps that folks generally don’t run on their ultraportables and netbooks.

That gives us a decent glimpse into the performance of Zacate’s dual Bobcat CPU cores. What about the APU’s graphics component?

Gaming

Call of Duty 4: Modern Warfare
Infinity Ward’s first Modern Warfare title is growing somewhat long in the tooth, but it still has a strong following in multiplayer circles. More importantly, it’s a good representative of the type of game you might want to play on a notebook that lacks serious GPU horsepower: not too old, but not too new, either. We tested Call of Duty 4 by running a custom timedemo, first at 800×600 with the lowest detail options, then again at 1366×768 with everything cranked up except for v-sync, antialiasing, and anisotropic filtering, which were all left disabled. (With the Eee PC, we opted for the 1024×600 native resolution instead of 1366×768, and with the Zbox HD-ND22, we were only able to use a resolution of 1360×768.)

Oh my. Although Zacate falls a reasonable distance behind our Toshiba Nile notebook in CPU tests, it speeds past in Modern Warfare. The only system in the same class that comes close is the Zbox, which is powered by a first-generation Nvidia Ion chipset.

Far Cry 2
Ubisoft’s safari-themed shooter has much more demanding graphics than CoD4, so it should really make our notebooks sweat. We selected the “Action” scene from the game’s built-in benchmark and ran it in two configurations; first at 1366×768 in DirectX 10 mode with detail cranked up, then at that same resolution in DX9 mode with the lowest detail preset. Vsync and antialiasing were left disabled in both cases. (Again, the Eee PC was run at 1024×600, since that’s the highest resolution its display supports, and the Zbox was run at 1366×760.)

We see a similar picture in Far Cry 2, where the Zacate and Zbox basically end up neck-and-neck.

That’s it for our conventional gaming benchmarks. Next up: some freestyle game testing.

Off the beaten path
Scientific benchmarks or not, we like to install different games on our laptops and manually tweak the options to see how well they run. A little subjective testing never hurt anybody, right?

I kicked off my freestyle gaming tests with DiRT 2, a long-time TR favorite and one of the better racing games out on the PC. At 1366×768 with the “low” preset, the demo’s Morocco track unfurled at a solid 20 FPS, give or take two or three. Frame rates dropped into the low teens upon crashes, but the game was surprisingly smooth and playable overall.

Next up was Left 4 Dead 2, which I ran at 1366×768 with trilinear filtering, no antialiasing, high shader detail, and medium effect, model, and texture detail. In the first map of the Dead Center campaign, frame rates ranged from a low of about 13 to a high of 36 FPS. From a seat-of-the-pants perspective, the game was completely playable despite notable choppiness during heavy action. Those massive zombie swarms aren’t easy on low-end hardware, but Zacate did a reasonably good job of keeping things smooth.

Feeling emboldened by these good results, I tried the Just Cause 2 demo. Things didn’t go so well there. At 800×600 with all the detail options turned down, the Zacate system yielded frame rates in the 18-20 FPS range in the first town. That’s just not good enough for a fast-paced action game, unfortunately. I had trouble shooting and driving straight, wishing there were somehow a way to turn the graphics detail further down.

I ended my freestyle game testing with a short round of Alien Swarm, specifically the single-player training level. At 1366×768 with antialiasing off, trilinear filtering, low shader and effect detail, and medium model and texture detail, the game chugged along with highs of nearly 30 and lows in the 15-17 FPS range during heavy action. I’d say the game was definitely playable overall, and it didn’t look half bad, as you can see above.

Video playback
I tested video decoding performance by playing the Iron Man 2 trailer in a variety of formats. Windows Media Player was used in full-screen mode for the H.264 QuickTime clips, while Firefox was used for the windowed YouTube test. In each case, I used Windows 7’s Performance Monitor to record minimum and maximum CPU utilization for the duration of the trailer.

	CPU utilization	Result
Iron Man 2 H.264 480p	9-40%	Perfect
Iron Man 2 H.264 720p	4-53%	Perfect
Iron Man 2 H.264 1080p	6-62%	Perfect
Iron Man 2 YouTube 720p windowed	32-82%	Choppy

Rest assured, Zacate’s UVD3 video decoding logic does a terrific job with H.264 content in Windows Media Player. Playing back high-definition Flash clips in Firefox turned out to be another story, though—there were quite a lot of dropped frames, even if I tried running the video in full-screen mode.

I asked the AMD folks about this, and they told me they’d been able to get smooth high-def Flash playback out of one of the same test systems without issue. Puzzled, I tried updating Firefox and the Flash plug-in to the latest versions. No dice. After some more prodding, I learned that AMD had run its internal tests in Internet Explorer. Sure enough, the same video played back as smooth as silk in Internet Explorer 8 with the latest Flash ActiveX plug-in. Interesting. I’d probably chalk this issue up to immature drivers. Hopefully, final Zacate systems will be able to deliver smooth Flash video in any browser.

Temperatures and power consumption
I didn’t bring an infrared thermometer or a power meter along, but AMD was kind enough to provide some, so I took the bait and got some readings while running the benchmarks you saw on the previous pages.

First, I amused myself with the infrared thermometer, pointing the beam at various surfaces (and the inside of my mouth, when the AMD guys weren’t looking). I measured 27.4°C at the base of the APU heatsink with the system at idle and 32.5°C during our Far Cry 2 benchmark’s third consecutive run. The fan speed seemed pretty much constant, and it was barely louder than a whisper.

I also checked the power meter while the Far Cry 2 test was finishing: it reported 33.6W draw at the wall. In x264, a purely CPU-bound benchmark, power draw at the wall was about 26W. Keep in mind these figures are for the entire system and were taken upstream of the power adapter, which AMD told me has an efficiency of around 83%.

Conclusions
AMD has pulled off quite an impressive feat with Zacate. On the CPU front, the dual Bobcat cores clearly play in the same league as Intel’s entry-level CULV and CULV 2010 processors. That means performance largely acceptable for everyday tasks, from web surfing to file compression. Even enthusiasts using an ultraportable or a nettop as their second or third PC ought to find this kind of performance acceptable.

On the graphics front, meanwhile, Zacate succeeds in outpacing both previous AMD solutions and current Intel ones by a fair margin. The only system that came close was our Zotac nettop, which has an Nvidia Ion integrated graphics chipset. You’ll be hard-pressed to find any ultraportable notebooks toting that kind of hardware these days, though; the closest thing you’d probably find would be something with a discrete, next-generation Ion GPU, and that’s a different class of hardware—nothing like the elegant two-chip Zacate-Hudson combo AMD will offer as part of the Brazos platform.

As I said last time, the only question that remains is battery life. If Zacate manages to match or exceed current solutions in terms of run time, which seems entirely possible considering the Brazos platform’s very spartan power draw, then AMD might just end up with the most attractive ultraportable platform on the market early next year. That would be a refreshing first, after the misses and near-misses we’ve seen from the company so far.

Of course, Intel isn’t asleep at the switch. With such heated competition from its chief rival, perhaps it won’t hesitate to whip up some Sandy Bridge-based CULV 2011 solutions, which could have considerably better graphics performance than current, CULV 2010 offerings. And the next generation of Atom processors, also due next year, might give netbooks some much-needed oomph. There’s no telling whether Intel will be able to reach the sweet spot of power efficiency and performance AMD appears to have achieved with Zacate, though.