Review sites like TR are a tricky business, let me say up front. We work constantly with the largest makers of PC hardware in order to bring you timely reviews of the latest products. Making that happen, and keeping our evaluations fair and thorough, isn’t easy in the context of large companies engaging in cutthroat competition over increasingly complex technologies.
I know for a fact that many folks who happen across TR’s reviews are deeply skeptical about the whole enterprise, and given everything that goes on in the shadier corners of the web, they have a right to be. That said, we have worked very hard over the years to maintain our independence and to keep our readers’ interests first among our priorities, and I think our regular audience will attest to that fact.
At its heart, the basic arrangement that we have with the largest PC chip companies is simple. In exchange for early access to product samples and information, we agree to one constraint: timing. That is, we agree not to post the product information or our test results until the product’s official release.
That’s it, really.
There are a few other nuances, such as the fact that we’re released from that obligation if the information becomes public otherwise, but they only serve to limit the extent of the agreement.
In other words, we don’t consent to any other constraint that would compromise our editorial independence. We don’t guarantee a positive review; we don’t agree to mention certain product features; and we certainly don’t offer any power over the words we write or the results we choose to publish. In fact, by policy, these companies only get to see our reviews of their products when you do, not before.
If you’re familiar with us, we may be covering well-trodden ground here, but bear with me. Our status as an independent entity is key to what we do. Most of the PR types we work with tend to understand that fact, so we usually get along pretty well. There’s ample room for dialog and persuasion about the merits of a particular product, but ultimately, we offer our own opinions. In fact, the basic arrangement we have with these firms has been the same for most of the 13 years of our existence, even during the darkest days of Intel’s Pentium 4 fiasco.
You can imagine my shock, then, upon receiving an e-mail message last week that attempted to re-write those rules in a way that grants a measure of editorial control to a company whose product we’re reviewing. What AMD is doing, in quasi-clever fashion, is attempting to shape the content of reviews by dictating a two-stage plan for the release of information. In doing so, they grant themselves a measure of editorial control over any publication that agrees to the plan.
In this case, the product in question is the desktop version of AMD’s Trinity APUs. We received review samples of these products last week, with a product launch date set for early October. However, late last week, the following e-mail from Peter Amos, who works in AMD’s New Product Review Program, hit our inbox:
We are allowing limited previews of the embargoed information to generate additional traffic for your site, and give you an opportunity to put additional emphasis on topics of interest to your readers. If you wish to post a preview article as a teaser for your main review, you may do so on September 27th, 2012 at 12:01AM EDT.
The topics which you are free to discuss in your preview articles starting September 27th, 2012 at 12:01AM EDT are any combination of:
– Gaming benchmarks (A10, A8)
– Speeds, feeds, cores, SIMDs and branding
– Experiential testing of applications vs Intel (A10 Virgo will be priced in the range of the i3 2120 or i3 3220)
– Power testing
We believe there are an infinite number of interesting angles available for these preview articles within this framework.
We are also aware that your readers expect performance numbers in your articles. In order to allow you to have something for the preview, while maintaining enough content for your review, we are allowing the inclusion of gaming benchmarks.
By allowing the publication of speeds, feeds, cores, SIMDs and branding during the preview period, you have the opportunity to discuss the innovations that AMD is making with AMD A-Series APUs and how these are relevant to today’s compute environment and workloads.
In previewing x86 applications, without providing hard numbers until October [redacted], we are hoping that you will be able to convey what is most important to the end-user which is what the experience of using the system is like. As one of the foremost evaluators of technology, you are in a unique position to draw educated comparisons and conclusions based on real-world experience with the platform.
The idea here is for AMD to allow a “preview” of the product that contains a vast swath of the total information that one might expect to see in a full review, with a few notable exceptions. Although “experiential testing” is allowed, sites may not publish the results of non-gaming CPU benchmarks.
The email goes on to highlight a few other features of the Socket FM2 platform before explaining what information may be published in early October:
The topics which you must be held for the October [redacted] embargo lift are:
– Non game benchmarks
The email then highlights each of these topic areas briefly. Here’s what it says about the temporarily verboten non-gaming benchmarks:
Non game benchmarks
– Traditional benchmarks are designed to highlight differences in different architectures and how they perform. We understand that this is a useful tool for you and that your readers expect to see this data. The importance of these results is in your evaluation, as the leading experts, of what these performance numbers mean. We encourage you to use your analysis if you choose to publish a preview article and if you find that to be appropriate to your approach to that article. The numbers themselves must be held until the October [redacted] embargo lift. This is in an effort to allow consumers to fully comprehend your analysis without prejudging based on graphs which do not necessarily represent the experiential difference and to help ensure you have sufficient content for the creation of a launch day article.
Now, we appreciate that AMD is introducing this product in an incredibly difficult competitive environment. We’re even sympathetic to the idea that the mix of resources included in its new APU may be more optimal for some usage patterns, as our largely positive review of the mobile version of Trinity will attest. We understand why they might wish to see “experiential testing” results and IGP-focused gaming benchmarks in the initial review that grabs the headlines, while burying the CPU-focused benchmarks on a later date. By doing so, they’d be leading with the product’s strengths and playing down its biggest weakness.
And it’s likely to work, I can tell you from long experience, since the first article about a subject tends to capture the buzz and draw the largest audience. A second article a week later? No so much. Heck, even if we hold back and publish our full review later (which indeed is our plan), it’s not likely to attract as broad a readership as it would have on day one, given the presence of extensive “previews” elsewhere.
Yes, AMD and other firms have done limited “preview” releases in the past, where select publications are allowed to publish a few pictures and perhaps a handful of benchmark numbers ahead of time. There is some slight precedent there.
But none of that changes the fact that this plan is absolutely, bat-guano crazy. It crosses a line that should not be crossed.
Companies like AMD don’t get to decide what gets highlighted in reviews and what doesn’t. Using the review press’s general willingness to agree on one thing—timing—to get additional control may seem clever, but we’ve thought it over, and no. We’ll keep our independence, thanks.
The email goes on to conclude by, apparently, anticipating such a reaction and offering a chance for feedback:
We are aware that this is a unique approach to product launches. We are always looking at ways that we can work with you to help drive additional traffic to your articles and effectively convey the AMD message. We strive to provide the best products in their price points, bringing a great product for a great price. Please feel free to provide feedback on what you find, both with the product and with your experience in the AMD New Product Review Program. We try to ensure that we are providing you what you need and appreciate any feedback you have to offer on how we can do better.
I picked up the phone almost immediately after reading this paragraph and attempted to persuade both Mr. Amos and, later, his boss that this plan was not a good one. I was told that this decision was made not just in PR but at higher levels in the company and that my objections had been widely noted in internal emails. Unfortunately, although fully aware of my objections and of the very important basic principle at stake, AMD decided to go through with its plan.
Shame on them for that.
It’s possible you may see desktop Trinity “previews” at other websites today that conform precisely to AMD’s dictates. I’m not sure. I hope most folks have decided to refrain from participation in this farce, but I really don’t know what will happen. I also hope that any who did participate will reconsider their positions after reading this post and thinking about what they’re giving up.
And I hope, most of all, that the broader public understands what’s at stake here and insists on a change in policy from AMD.
If this level of control from companies over the content of reviews becomes the norm, we will be forced to change the way we work the firms whose products we review. We will not compromise our independence. We believe you demand and deserve nothing less.
Update: AMD has issued a statement on this matter.