The mother of all 99th %ile FPS per dollar scatter plots?

From the pixels, bits, and shaders to the graphic cards that power them. Discuss the latest from AMD and NVIDIA here.

Moderators: morphine, SecretSquirrel

The mother of all 99th %ile FPS per dollar scatter plots?

Postposted on Wed Dec 04, 2013 2:56 pm

I find myself using a certain GPU hierarchy table frequently and I would love to have something similar from TR. Would it be possible to get a scatter plot for all tested cards with filters? Would TR be willing to put all their findings in a Google Doc or similar format?
gigafinger
Gerbil
Silver subscriber
 
 
Posts: 51
Joined: Fri Feb 24, 2012 8:53 am
Location: Derry, NH

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Wed Dec 04, 2013 3:10 pm

We test different classes of cards differently, since they're really most appropriately used at different resolutions and quality levels. Over time, things change that affect performance, like game patches, newer games being added to our test suite, and new video drivers.

So the short answer is that doing the sort of real-world testing we do across a broad range of graphics cards in a way that offers truly consistent, up-to-date comparisons isn't really feasible.

As close as we come is something like the scatter plots here:

http://techreport.com/review/25602/amd- ... reviewed/9

That covers a broad range of the latest cards from AMD and Nvidia, along with older Radeons in one segment of the market. Going beyond that with real-world 99th percentile frame times.... is a very, very big project.
Scott Wasson - "Damage"
Editor - The Tech Report
Damage
TR Staff
Gold subscriber
 
 
Posts: 1699
Joined: Wed Dec 26, 2001 7:00 pm
Location: Lee's Summit, Missouri, USA

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Wed Dec 04, 2013 3:39 pm

I find myself looking at a GPU heirarchy table from another site as well.

Is there a shortcut TR could take? Limit it to one game, only run the test once per card for that game, etc.?
Milo Burke
Gerbil Team Leader
Gold subscriber
 
 
Posts: 236
Joined: Thu Nov 07, 2013 11:49 am

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Wed Dec 04, 2013 4:05 pm

I think what they're asking for is a Tech Report GPU hierarchy table so they don't have to rely on the Tom's idiots. I know I'd trust TR a lot more on a table like that. Probably doesn't even need to have numbers.
Main: i5-3570K, ASRock Z77 Pro4-M, Asus GTX660 TOP, 120 GB Vertex 3 Max IOPS, 2 TB Samsung EcoGreen F4, 8GB 1600MHz G.Skill @1.25V, Silverstone PS07B
HTPC: A8-5600K, MSI FM2-A75IA-E53, 4TB Samsung SSHD, 8GB 1866MHz G.Skill, Hand-Built Wood Case
DPete27
Gerbil Jedi
Silver subscriber
 
 
Posts: 1732
Joined: Wed Jan 26, 2011 12:50 pm
Location: Madison, Wisconsin

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Wed Dec 04, 2013 4:28 pm

Damage wrote:Blah blah blah... is a very, very big project.

What's the matter Col. Sanders? Chicken?

I agree though, but there has to be some why I can leverage all your hard work.
gigafinger
Gerbil
Silver subscriber
 
 
Posts: 51
Joined: Fri Feb 24, 2012 8:53 am
Location: Derry, NH

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Wed Dec 04, 2013 8:20 pm

I have to admit, it'd be nice to see a chart with some older gen stuff up against the latest greatest.

Like nobody has a GTX 480 tested against 700 series cards, for example.
Kougar
Gerbil XP
 
Posts: 433
Joined: Tue Dec 02, 2008 2:12 am
Location: Texas

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Thu Dec 05, 2013 7:15 am

This is about the best option:
http://www.videocardbenchmark.net/gpu_list.php

The problem with a test that covers such a huge range of graphics cards is that it has to be ancient to run on all those old DX9 cards from last decade.
Take the results with a huge pinch of salt, assume the actual rankings have anything up to a 50% margin of error.
I hear rig lists are all the rage, and I <3 the rage! Workstation = Black tower thing; HTPC = Shhhh!; Laptop - AMAZING FOLDING PC!
Chrispy_
Minister of Gerbil Affairs
Gold subscriber
 
 
Posts: 2157
Joined: Fri Apr 09, 2004 3:49 pm

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Thu Dec 05, 2013 7:25 am

Chrispy_ wrote:This is about the best option:
http://www.videocardbenchmark.net/gpu_list.php

No. Videocardbenchmark.net and CPUBenchmark.net are to be avoided at all costs, along with Notebookcheck.net.

For what it's worth, while I'd appreciate something like this, I find the Tom's chart to be close enough for government work. Anytime you're talking about a purchase close enough where you need more accurate data, you can usually find the two cards matched up in a review somewhere.

You can also extrapolate numbers on similar cards using the clock rates, shader counts, and memory bandwidth. I find Wikipedia to be an invaluable resource for this. Check these articles:

Comparison of Nvidia GPUs lists basically all Geforces ever, and has the relevant data.

Comparison of AMD GPUs does the same with Radeons.
i7-4790K (Seidon 120XL), 2x8GB DDR3-1866 CL9 1T, GA-Z97MX-GAMING 5, SAPPHIRE TRI-X OC 290X, 128GB Vertex 4 / 4x60GB Vertex Plus R2, SST-DA1000 (PSU), 2x VS229H-P, 1x VG248QE, 1x MIMO 720F, Corsair Vengeance K90+M95
auxy
Gerbil Elite
 
Posts: 813
Joined: Sat Jan 19, 2013 4:25 pm
Location: the armpit of Texas

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Thu Dec 05, 2013 9:06 am

I don't think anything can be done on "performance per dollar" on older cards. New games stress older cards, new CPUs and RAM can inflate the results of newer cards, and once a card is replaced and taken off the market, it renders any value proposition null. The best you can do is compare what's under the hood and estimate where older cards fall, as Auxy suggested, but keep in mind that it would be nothing more than an estimate (other updates could further degrade performance), and it would only serve people who have those older cards.
Damage wrote:Don't try to game the requirements by posting everywhere, guys, or I'll nuke you from space.

-Probably the best Damage quote ever.
superjawes
Graphmaster Gerbil
Gold subscriber
 
 
Posts: 1160
Joined: Thu May 28, 2009 9:49 am

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Thu Dec 05, 2013 10:20 am

auxy wrote:along with Notebookcheck.net.


Aren't they at least useful for telling whether your WTForce MX680MX2-M is weaker or stronger than some card you can actually find benches for?
Game_boy
Gerbil Elite
 
Posts: 564
Joined: Mon Aug 06, 2007 12:46 pm

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Thu Dec 05, 2013 11:21 am

My GPU history:

Code: Select all
Model     Paid   Date
X800Pro   $250   7/2005
8800GTS   $310   8/2007
HD 4870   $270   8/2008
HD 7950   $400   6/2012


I think the performance per dollar is important. Maybe not for seeing if an old card is viable in a modern build, but for upgrade cost effectiveness. My 4870 lasted me 4 years and is still on par with the lesser modern cards (HD 7750, GTX650). Spending $270 for 4+ years of PC gaming is pretty good (my nephew is running the 4870 now). As far as I can tell, the 7950 should carry me quite a ways too. It's on the same tier as the current R9 270/280 and GTX 670/760 cards. I'd like to see where the 7950 and the latest cards match up on the scatter plots. I blame TR for making them so damn useful :wink:

Maybe performance per dollar per game would be a decent method of getting all TRs data into "master plots?" If a card was tested for Farcry 3, put it on a Farcy 3 scatter plot. There's got to be some game overlap in the collected data.
gigafinger
Gerbil
Silver subscriber
 
 
Posts: 51
Joined: Fri Feb 24, 2012 8:53 am
Location: Derry, NH

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Thu Dec 05, 2013 11:35 am

Game_boy wrote:
auxy wrote:along with Notebookcheck.net.


Aren't they at least useful for telling whether your WTForce MX680MX2-M is weaker or stronger than some card you can actually find benches for?

No. The results are user-submitted and could be falsified or just straight-up wrong.

It's easy to tell how a notebook GPU compares to a desktop model. Just look at the specifications in the wikipedia links I just gave. Compare the clocks, shader counts, and memory bandwidth, and that will give you a very good estimation of the performance of the chip.
i7-4790K (Seidon 120XL), 2x8GB DDR3-1866 CL9 1T, GA-Z97MX-GAMING 5, SAPPHIRE TRI-X OC 290X, 128GB Vertex 4 / 4x60GB Vertex Plus R2, SST-DA1000 (PSU), 2x VS229H-P, 1x VG248QE, 1x MIMO 720F, Corsair Vengeance K90+M95
auxy
Gerbil Elite
 
Posts: 813
Joined: Sat Jan 19, 2013 4:25 pm
Location: the armpit of Texas

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Thu Dec 05, 2013 11:46 am

Nope. We want a comparison chart. =]
Milo Burke
Gerbil Team Leader
Gold subscriber
 
 
Posts: 236
Joined: Thu Nov 07, 2013 11:49 am

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Thu Dec 05, 2013 11:55 am

gigafinger wrote:My GPU history:

Code: Select all
Model     Paid   Date
X800Pro   $250   7/2005
8800GTS   $310   8/2007
HD 4870   $270   8/2008
HD 7950   $400   6/2012


I think the performance per dollar is important. Maybe not for seeing if an old card is viable in a modern build, but for upgrade cost effectiveness. My 4870 lasted me 4 years and is still on par with the lesser modern cards (HD 7750, GTX650). Spending $270 for 4+ years of PC gaming is pretty good (my nephew is running the 4870 now). As far as I can tell, the 7950 should carry me quite a ways too. It's on the same tier as the current R9 270/280 and GTX 670/760 cards. I'd like to see where the 7950 and the latest cards match up on the scatter plots. I blame TR for making them so damn useful :wink:

Maybe performance per dollar per game would be a decent method of getting all TRs data into "master plots?" If a card was tested for Farcry 3, put it on a Farcy 3 scatter plot. There's got to be some game overlap in the collected data.

Just so my earlier point was clear, performance per dollar [P/$] is important, but the metric has a shelf life. Once a card is off the market, it doesn't do any good comparing its performance per dollar because the prices are off or nonexistent. If you compared the 4870's P/$ (from 2008) to the 7950's (from 2012), the 4870 would look awful. In order to maintain an accurate P/$ scatter plot with these older cards, you would not only need to estimate the performance, but the price as well. What's a fair price for a 2008 card in new condition?

It would be worth it to estimate and compare the performance of both, since there are plenty of people who could be upgrading, but few people are going to buying a NEW 4870 in 2013, and virtually no one will buy one at its 2008 price.
Damage wrote:Don't try to game the requirements by posting everywhere, guys, or I'll nuke you from space.

-Probably the best Damage quote ever.
superjawes
Graphmaster Gerbil
Gold subscriber
 
 
Posts: 1160
Joined: Thu May 28, 2009 9:49 am

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Fri Dec 06, 2013 9:56 am

@super: I think its safe to say that the majority of users are going to buy either a current gen or 1-previous gen GPU as an upgrade. TRs current reviews already show the price/performance comparisons there. And I'd hope anyone who owns an older card that they're looking to upgrade knows (or can look up) what they paid for it.
Main: i5-3570K, ASRock Z77 Pro4-M, Asus GTX660 TOP, 120 GB Vertex 3 Max IOPS, 2 TB Samsung EcoGreen F4, 8GB 1600MHz G.Skill @1.25V, Silverstone PS07B
HTPC: A8-5600K, MSI FM2-A75IA-E53, 4TB Samsung SSHD, 8GB 1866MHz G.Skill, Hand-Built Wood Case
DPete27
Gerbil Jedi
Silver subscriber
 
 
Posts: 1732
Joined: Wed Jan 26, 2011 12:50 pm
Location: Madison, Wisconsin

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Fri Dec 06, 2013 2:31 pm

There's more to it than just knowing which of the new cards is the best value for performance. I also want to know how significant the upgrade is from what I already have (or what my friends have). And, naturally, I'm curious how much performance has improved in the last three, six, and nine years. :)
Milo Burke
Gerbil Team Leader
Gold subscriber
 
 
Posts: 236
Joined: Thu Nov 07, 2013 11:49 am

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Sat Dec 07, 2013 8:10 am

I'd honestly just be happy with a single chart with old and new hardware on it. I can do the cost / perf myself based on current pricing, and once I decide on a GPU I could then go back to the original review article for 99th percentile graphs. Basically it all comes back to needing a single unified graph with a wide range of hardware generations on it to start with.
Kougar
Gerbil XP
 
Posts: 433
Joined: Tue Dec 02, 2008 2:12 am
Location: Texas

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Sat Dec 07, 2013 4:19 pm

I have to admit that I'm with Auxy here:

I use Wikipedia an awful lot to work stuff out - one you know the clocks, architecture and shader/core count, you can extrapolate pretty much anything based on common cards that get reviewed.
I hear rig lists are all the rage, and I <3 the rage! Workstation = Black tower thing; HTPC = Shhhh!; Laptop - AMAZING FOLDING PC!
Chrispy_
Minister of Gerbil Affairs
Gold subscriber
 
 
Posts: 2157
Joined: Fri Apr 09, 2004 3:49 pm

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Sun Dec 08, 2013 3:06 am

I've mentioned something like this before, so you can have a direct comparison across all graphics cards and processors no matter how old or new they are. I also understand where Scott is coming from with the different setups.

Have you guys considered using a standardized test and throwing it into all the testing suites no matter what? It wouldn't mean you'd need to standarize all your testing, but could have one static variable that wouldn't change over the years.

Say for instance, testing 3D Mark 11 at 1080p or whatever setting and always having that available in the article. That'd allow you guys to build a database off of it and also allow people to directly compare older and newer hardware even if the test isn't completely representative of real world test scenarios people run into every day, it'd allow people to form correlations based on data in whatever review is being done and newer reviews. Since it's highly automated I don't think it'd add that much extra testing (or time). You could continue to update such a database with each article, continually adding to it.

You'd still run into a couple discrepancies, like using different processors, memory, and/or graphics cards, but the tests are pretty compartmentalized to test only certain things (like graphics or processor).
Bensam123
Gerbil Elite
 
Posts: 952
Joined: Wed May 29, 2002 12:19 pm

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Sun Dec 08, 2013 4:57 am

I prefer using 'real-world' tests to component comparisons. What I did when selecting my last purchase was find the review of the latest graphics card release on TechPowerUp and then look at the charts on the Performance Summary page. E.g.

Image

http://www.techpowerup.com/reviews/NVIDIA/GeForce_GTX_780_Ti/27.html

Then I find a cards that offer a reasonable performance increase and then I start looking up prices on Google Shopping.

The nice thing is they test around 15 games and they test at various popular resolutions so you can just look specifically at performance at your monitor's resolution. If your current card isn't on the chart just find an older review's chart with it on and you can use that together with a new review's chart to roughly extrapolate your performance gain. Unfortunately they don't test frame times though so once you've found a card that fits the bill look it's review on TR. ;)
Intel Core i5-4670K | Asus Z87-A | G.Skill 8GB 2400MHz CL10 | Asus DirectCU II R9 290 4GB | Samsung 840 120GB |Thermalright Macho | Lancool PC-K59
puppetworx
Gerbil Elite
Silver subscriber
 
 
Posts: 515
Joined: Tue Dec 02, 2008 5:16 am

Re: The mother of all 99th %ile FPS per dollar scatter plots

Postposted on Sun Dec 08, 2013 9:40 am

I was trying to keep it simple and feasible for the way TR likes testing their cards (which is different based on what it is).
Bensam123
Gerbil Elite
 
Posts: 952
Joined: Wed May 29, 2002 12:19 pm


Return to Graphics

Who is online

Users browsing this forum: No registered users and 4 guests