We try not to get too much into behind-the-scenes drama here at TR, "inside baseball" stuff like how we deal with public relations types, because we think the focus should be on the products, test results, and reader concerns. I think that’s generally the right thing to do, but today I’m going to make an exception to talk a little bit about some recent happenings.
As you might imagine, both Geoff and I have spent a tremendous portion of the past month-plus focusing on new products from AMD, including the Radeon HD 3800 series, the 790FX chipset, and the Phenom processor. There’s been much to cover, and we’re generally quite happy to see intriguing new products come our way. But this three-for-one launch has been a less-than-pleasant experience for reasons that both inconvenience us professionally and, to some extent, affect our readers as consumers, too.
I should begin by saying that we have close working relationships with companies like AMD, Intel, and Nvidia—a necessity given our tightly focused coverage of the core of the PC. That sometimes leads us to get information about upcoming products months in advance, so we can plan our editorial schedules and prepare for product reviews in advance. Such was the case with the three elements of AMD’s "Spider" platform. I visted AMD’s Toronto offices back in late September for an early briefing on these products, and Geoff followed up by attending a larger press event in New York in early November. We had prepared well ahead of time review these three products when they arrived.
To give you some idea, in my case, making those preparations involved weeks of work, believe it or not. I revamped our entire CPU test suite in preparation for the Phenom. The first fruits of this effort were seen in our Core 2 Extreme QX6850 review, which included brand-new everything: software revisions for a large proportion of our test suite, a new P35-based mobo for Intel CPUs with DDR3 memory, an upgrade to 4GB of RAM for all test systems, all new WD Caviar SE16 hard drives, new BIOS and driver revisions all around, and a number of new applications and games for testing. We acquired this stuff, rebuilt our test systems from scratch, and began accumulating results for what became ten different competing CPU speed grades for comparison to the Phenom.
I made similar preparations for the Radeon HD 3800 series, including the addition of new power supplies, new Caviar SE16 drives, a new X38-based CrossFire test platform, and a from-the-ground-up software rebuild involving the very latest new games. Likewise, Geoff spent substantial time reworking his chipset test setup in anticipation of the 790FX.
We attended these briefings and did this prep work in anticipation of something we’d come to expect from long experience in working with AMD on reviews over the years, something we’d been promised by AMD: samples of the actual products to test and review. That is, of course, the heart and soul of our enterprise, to tell you how these things measure up in enough detail that you’re well-informed about the products. We prepare ahead of time as much as possible, because experience has taught us that things often change at the last minute: launch dates get pulled ahead or pushed back, products get changed, key specs are revised, and hardware gets delayed. As a result, we oftentimes have a very small window of time in which to test a new video card or processor, and we want to be sure to meet the standards we’ve established for thoroughness and detail if at all possible.
Unfortunately, as the launch dates approached for AMD’s new GPUs, CPUs, and chipset, we slowly started to realize that not all was well. The first major indicator was the fact that AMD chose to send out a pair of Radeon HD 3850 cards for review, but neglected to include 3870s. Then, just as Geoff was returning from the NYC press briefing, we received an email inviting us to another press event in Lake Tahoe.
This threw us for a loop.
Why, we wondered, would we be attending a third event for this round of products? Had we not endured enough PowerPoint time already? The answer from AMD PR was even worse than we thought: Tahoe would be a "hands-on" testing event, and would likely be our only chance to spend time with the hardware before the product launch.
Now, given what I’ve said about the preparation we’d done ahead of time in our labs, you can imagine the relative value of a one-day test session, completely off site, where we work with systems configured by AMD. Put bluntly, it’s almost entirely worthless to generate the kind of comparative test data we put into our reviews. Just setting up a test rig can take us as long as one of these events lasts, and that’s assuming you have complete freedom to swap components around at will. These events don’t afford such freedom.
More importantly, learning that AMD was looking to restrict reviewers’ access to its products before the launch was the equivalent of seeing a bright, red flare shooting upward into the night sky. As a reviewer, I’ve become conditioned over time to see this sort of PR move as a classic and sure-fire indicator of a poor product.
Now, I’m not saying that we here at TR or those in the press in general have a divine right to advance review samples of products. That’s not my position. But I’ve observed over the years that would-be clever PR people have an uncanny tendency to change the terms on which they work with us, to change their philosophy about product samples, at the precise time when the product they’re shepherding to market has the moldy odor of a possible stinker emanating from it. We’ve seen this dynamic unfold time and again, yet the PR types—whose average tenure tends to be measured in months rather than years—continue to pull out this tactic at key moments, as if it were smart, new, and likely to succeed.
Of course, that simply doesn’t work with us, for a variety of reasons. First and foremost among them is this one: we will review the product, one way or another. When we called the GeForce FX a disappointment early on and nicknamed the cooler a "Dustbuster", Nvidia excommunicated us for a year. So we paid the ransom for one on eBay and reviewed it anyway, complete with decibel-meter readings for that ridiculous cooler.
We choose to work with companies like AMD only because it suits our purposes, which are to bring fair and thorough reviews to our readers in as timely a manner as possible. Generally, we’ve found that playing nice with chipmakers has been the best path to that goal, even though we often have to agree to wait for launch day to publish a review. But we don’t have to make that deal, and I see no reason to do so if we can’t get our hands on the product. We could always go the "gray market" route, acquiring a chip from other sources and publishing test results as soon as possible. In many cases, that would mean having hard numbers before the launch date, even if they were from pre-release hardware.
We’re also not going to be dazzled by a trip to Tahoe to see the sights. We’ve long since learned that hotel conference rooms with PowerPoint slides look the same whether they’re in San Jose, Tahoe, or Tunisia. The sight we want to see is review hardware humming away on our test bench, and nothing else. We’re geeks, you see.
In the case of the Spider platform, AMD’s reluctance to provide review samples had less impact on us than it did on some publications. We managed to wrangle at least one Radeon HD 3870 for testing, and we got our hands on a couple of 790FX-based motherboards and a Phenom engineering sample. But our Phenom review was a day late, and we wound up testing early revisions of the Asus M3A32-MVP Deluxe motherboard and the Phenom processor that don’t represent final products—or so we hope, since the user experience was decidedly less than ideal. We’re currently engaged in follow-up work, hoping to confirm that the shipping hardware works properly.
Of course, we try very hard to separate a company’s shaky PR efforts from the products themselves. We owe it to our readers and to the smart engineers who work behind the scenes on these things to get our evaluations right, even if we’re exasperated by the press interface.
But AMD’s path to the Spider platform launch could have—and should have—been a much smoother one than it was, and it would have worked out better for all parties involved. AMD would have avoided communicating the messages it did to us and, by proxy, to the public. Among them: "we’re not confident in our products," "we’re not a particularly well-run organization," and "our chipset’s south bridge is probably broken." We would have been able to sleep more, worry less, and provide fuller coverage to our readers. And our readers would have been able to make better-informed decisions about AMD’s products on day one, understanding their best merits in the context of stable release hardware, including Phenom processors with working Cool’n’Quiet power-saving mechanisms.
Instead, what we got was a frustrating result in which the drawbacks of the Spider platform components are generally clearer than their virtues, and we’re left wondering whether AMD can be trusted to make things right. We’re also unsure about the value of working with AMD going forward, given the apparent change in PR philosophy. Things may be tough for a while, but AMD could learn a thing or two from how Intel handled its rough times. Even in the darkest days of the Prescott fiasco, Intel supplied reviewers with new product samples regularly. The grace and class with which Intel handled itself cultivated goodwill with PC enthusiasts, and that paid off when the time came for Intel to recover its competitiveness with Core 2 Duo. Things may be rather difficult for AMD for a period. The company would do well to handle its struggles with similar grace and class.