BACK IN OCTOBER OF 2001, when AMD launched its first Athlon XP processors with "model number" ratings, the company told us it would be working on an industry-wide initiative to produce a new standard for measuring performance. The model-number rating system was supposed to be temporary:

For the future, the company has big plans. AMD is promising "an industry-wide initiative to develop a new and more complete measure of performance that end-users can trust." This new performance metric is due for rollout in 2002.

But I had my doubts:

Until that new standard arrives, AMD has cooked up its own set of benchmarks to use. AMD calls the results from this suite of tests a "bridge metric." It is intentionally and overtly temporary (at least for now; these things tend to grow roots).

Now, with the end of 2002 rapidly approaching, those reservations look mighty prescient. AMD is still assigning model numbers to its processors, supposedly based on their performance compared to an older T-bird Athlon at a given clock speed (never has AMD said the performance rating actually targets Pentium 4 processors). At 2.25GHz, the latest Athlon XP is purportedly as fast as a T-bird at 2.8GHz. Trouble is, T-birds never ran any faster than half that speed, so AMD's comparison target is increasingly mythical.

Since last October, we've heard very little out of AMD about the new performance metric coming from its True Performance Initiative. I wasn't about to let 2002 slip by without finding out what became of TPI, so I started poking around asking nosy questions. AMD was kind enough to point me to the fortuitously named Hal Speed, the company's Senior Manager of Strategic Initiatives and head TPI honcho. I asked Hal all manner of questions like, "Is TPI still alive?" "When will we see it?" and "Is that really your real name, really?"

Talking speed
Mr. Speed (you've gotta love it) wasn't at liberty to answer all of my questions because of non-disclosure agreements between AMD and its TPI partners, but I was able to extract enough information from him to piece together an interesting picture of where AMD's initiative currently stands. I'll try to relate to you what I've learned.

First and foremost, I'm pleased to report that TPI is very much alive at AMD. The new performance metric won't see the light of day in 2002, but AMD expects to launch it publicly early next year. I asked Speed if the launch would be timed to coincide with the introduction of Hammer processors, and he said no. TPI will stand on its own, it seems.

AMD's secrecy about its effort to devise a new means of measuring performance seems a little counterintuitive. One would expect an open process, draft proposals, requests for comment, and all the other things that come with the forging of a new standard in the Internet era. Speed acknowledged that Intel's shadow looms large over this effort, and the fear of reprisal weighs on the minds of AMD's partners. However, he said there would be ample opportunity for additional parties to comment on the effort or join its ranks once the entire plan is announced.

Speed said the effort has changed since its inception, now that AMD has taken on more partners. The new metric now looks at overall system performance, not just CPU performance. The goal is to provide consumers with an easy way of understanding how a PC will perform in common, real-world applications. The problem TPI seeks to address isn't just the "MHz myth" but all that comes with it: unbalanced systems with nice-sounding specs that can't deliver when it matters. Speed's example was PCs with integrated graphics. These budget specials come with fast processors and claim to have 3D graphics capabilities, but when users get them home, even basic 3D games aren't likely to run well.

So will this new metric attempt to conjure a single number to convey overall performance or a set of numbers? Speed said it could be both. He was fuzzy on specifics, but he said consumers might have something to guide them in terms of overall performance, then something more specific to guide them about how a particular system would handle something they'd want to do, like 3D gaming.

Obviously, then, this won't be another SPEC. The focus is on consumer desktop systems, not servers or workstations, where buyers are more sophisticated about what drives system performance. For the same reasons, the effort will focus on Windows, not Linux or other operating systems.

Speaking of SPEC, I asked whether we would be witnessing the creation of another independent standards body for oversight of this new performance metric. After a difficult pause: "No comment." I'll let you interpret that one as you will.

The new metric's testing methodology will likely incorporate a mix of application and synthetic benches. Speed mentioned early in our conversation how AMD had joined BapCo, and he said the experience has been very positive for AMD. Sysmark 2003 should be a "much better" benchmark than the 2002 revision. However, he wouldn't confirm whether AMD would incorporate Sysmark into a suite of tests for this new metric. Speed expects the effort to use existing benchmarks whenever applicable, but they will see that voids get filled when no acceptable benchmarks exist.

So could The Tech Report, as a hardware review site, duplicate the testing and come up with a rating? Speed said that some algorithms may not be totally open, but "transparency is a goal" of the effort. We'll have to see how those conflicting realities get worked out.

Conundrums—and worse
So there you have it. We don't know all the details yet, but AMD's True Performance Initiative continues to percolate behind the scenes. All should be revealed some time early next year. I'd expect to see the foundation in place before Athlon 64 (Clawhammer) processors arrive, but that's just my hunch.

Incidentally, one of Speed's most interesting comments came as we wrapped up our conversation. I asked him about the possibility that the creation of a really good PC performance metric could hurt CPU makers. After all, a really well-balanced system might have 512MB of RAM, a fast hard drive, a good graphics card, and a processor no faster than, say, a Duron 1GHz. Would a faithful real-world performance metric sell more RAM and better graphics cards at the expense of faster CPUs? Speed answered by noting that sensitivity to processor performance depends on one's situation, which is true as far as it goes. He reiterated the new performance metric's goal to get more balanced systems to consumers.

He then restated the case for a trustworthy indicator of overall system performance, and noted that if the industry didn't do it, government would.

Now there's a scary thought.