Personal computing discussed

Moderators: renee, JustAnEngineer

 
tftio
Gerbil In Training
Topic Author
Posts: 6
Joined: Thu Jul 30, 2015 1:33 pm

Building a Titan X based CUDA workstation

Thu Jul 30, 2015 1:48 pm

Hey, everybody. My new job has me doing a lot of single-precision CUDA work, and I need to build a PC as a primary workstation. This will be the first time I've done this in more than a decade, so I'm feeling a little at sea here. As this is going to be the way I make my money, and I work from home, I'm interested in stability, then noise, and then everything else.

Questions:

1. Is there any reason to think that Skylake will have a Haskell-E processor type line any time in the near future? IOW, should I pull the trigger on an X99, or wait a few months for something EVEN MORE TERRIFYING?
2. How loud are the stock coolers on a Titan X? I see that EVGA has some -HYBRID version with a custom water cooling loop -- is that a noise solution, or a thermal one? I know the two are inextricably linked, but I'm more interested in a low noise floor at stock speeds than the ability to run much hotter

If I do go X99, there's no good reason to pay 2x the cost for a 5960 (as opposed to a 5930), right?

TIA!
 
maxxcool
Gerbil Elite
Posts: 855
Joined: Thu Sep 12, 2002 8:40 am
Location: %^&*%$$
Contact:

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 1:58 pm

This is one case where I (me personally) would shy away from a personal build. Go with a Dell build. get 'overnight oniste 24/hour hardware support' as part of the purchase. If this *IS* your bread and butter .. waiting for Newegg (for a replacement part or warranty) could cost you more money than the dell purchase itself in terms of raw income, reputation of reliability and availability.
Cybert said: Capitlization and periods are hard for you, aren't they? I've given over $100 to techforums. I should have you banned for my money.
 
tftio
Gerbil In Training
Topic Author
Posts: 6
Joined: Thu Jul 30, 2015 1:33 pm

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 2:52 pm

That makes a lot of sense; but I need that Titan. Or at least, I can justify the expense of a Titan over a 980 Ti because our applications are memory bound. I might be the ONLY PERSON ON EARTH who actually needs 12GB of VRAM. nVidia should be paying me!

The real problem with going with a prebuilt is that their nVidia workstations predominantly have Quadros, and I really don't need to pay out my nose for the double-precision compute premium. Also, I'm not made of stone -- I'll play games on it.
 
cobalt
Gerbil First Class
Posts: 171
Joined: Mon Oct 30, 2006 11:28 am

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 3:06 pm

tftio wrote:
That makes a lot of sense; but I need that Titan. Or at least, I can justify the expense of a Titan over a 980 Ti because our applications are memory bound. I might be the ONLY PERSON ON EARTH who actually needs 12GB of VRAM. nVidia should be paying me!

The real problem with going with a prebuilt is that their nVidia workstations predominantly have Quadros, and I really don't need to pay out my nose for the double-precision compute premium. Also, I'm not made of stone -- I'll play games on it.


There's no high-end Maxwell with fast double support -- Quadros don't have double-precision support either. If you're getting a high end card today with double precision support, it's a Kepler.

If they have a model that you can configure with a high-end GPU, you can just configure it with a low-end GPU and stuff your own Titan X in it. You should check to be sure, but as far as I know installing aftermarket cards doesn't affect your warranty.

(And, in fact, it may be a good idea to separate your display card from your compute card anyway.)
 
cobalt
Gerbil First Class
Posts: 171
Joined: Mon Oct 30, 2006 11:28 am

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 3:21 pm

tftio wrote:
1. Is there any reason to think that Skylake will have a Haskell-E processor type line any time in the near future? IOW, should I pull the trigger on an X99, or wait a few months for something EVEN MORE TERRIFYING?
2. How loud are the stock coolers on a Titan X? I see that EVGA has some -HYBRID version with a custom water cooling loop -- is that a noise solution, or a thermal one? I know the two are inextricably linked, but I'm more interested in a low noise floor at stock speeds than the ability to run much hotter

(3.) If I do go X99, there's no good reason to pay 2x the cost for a 5960 (as opposed to a 5930), right?


1. No. It's going to be like another year before a new -E series according the rumor mill. No idea if that's right, but the -E series has been lagging about that amount for several years now, so I see no reason to expect any differently this time. (Edit: I was thinking Skylake-E. I hadn't thought about a Broadwell-E showing up sooner, as suggested below. I don't know if it would be worth waiting for.)

2. The Hydro Copper version is overkill, so I think you're kind of stuck with the reference cooler. I don't see any other aftermarket coolers on newegg. The reference NVTTM cooler isn't silent, but it's supposedly a nice low frequency whooshing, and it blows air straight out of the case. It does have to ramp up the RPMs a little on hotter cards, so you'll see it be a little louder on the 980Ti/TitanX than on the base 980. Here are the TR review commentaries on the 980Ti / TitanX:
http://techreport.com/review/28356/nvid ... eviewed/12
http://techreport.com/review/27969/nvid ... eviewed/11

3. Well, it's 8 cores instead of 6. But it's also a lower clock speed (3.5 vs 3.0 base), so ignoring the price, it's a bit of a tradeoff. If you don't need 40 PCIe lanes (i.e. you're only using one Titan X), you could also save another $200 and get the i7-5820K, which takes only a little more clockspeed hit relative to the 5830K (3.3 vs 3.5), and it's higher than the 5960K. I think you'll need a Xeon if you want ECC support, though.
Last edited by cobalt on Thu Jul 30, 2015 3:32 pm, edited 1 time in total.
 
the
Gerbil Elite
Posts: 941
Joined: Tue Jun 29, 2010 2:26 am

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 3:23 pm

tftio wrote:
Hey, everybody. My new job has me doing a lot of single-precision CUDA work, and I need to build a PC as a primary workstation. This will be the first time I've done this in more than a decade, so I'm feeling a little at sea here. As this is going to be the way I make my money, and I work from home, I'm interested in stability, then noise, and then everything else.


As long as you're sure you need the single precision performance, then the Titan X is an excellent choice. If by chance you need double precision support or you will in the future, the vanilla Titan or the Titan black would be a better card for it.

tftio wrote:
Questions:

1. Is there any reason to think that Skylake will have a Haskell-E processor type line any time in the near future? IOW, should I pull the trigger on an X99, or wait a few months for something EVEN MORE TERRIFYING?


The answer is yes but Broadwell-E is looking to launch at the end of the year and utilize the X99 platform. From all indications, Broadwell-E will mimic the Sandybridge-E to Ivy Bridge-E change: not much to write home about. Expect a few 100 Mhz here and there, faster DDR4 support (officially) too.

SkyLake-E should be roughly ~12 months behind that in late 2016 or early 2017. The way Intel is updating their enthusiast/workstation platform, SkyLake-E will likely introduce a new socket and chipset. Current rumor is that future Xeon Phi and SkyLake-E chips will share a common socket so a 384 bit DDR4 memory interface is possible.

tftio wrote:
2. How loud are the stock coolers on a Titan X? I see that EVGA has some -HYBRID version with a custom water cooling loop -- is that a noise solution, or a thermal one? I know the two are inextricably linked, but I'm more interested in a low noise floor at stock speeds than the ability to run much hotter


I haven't heard a Titan X specifically but my experience with reference GTX970 and GTx 980 reference coolers (which is similar to the Titan X reference one) has been good. I wouldn't call them loud but I can hear them under load. For non-load scenarios, I'd have to go out of my way to hear them by getting closer etc. Your mileage may vary with different ears and acoustictastes.

tftio wrote:
If I do go X99, there's no good reason to pay 2x the cost for a 5960 (as opposed to a 5930), right?


The cheapest 6 core chip is neutered by only having 28 PCI-E lanes. In all honesty, I'd have preferred a 4 core chip with 40 PCI-E lanes for $300 to $400 as that is the primary driver of the X99 platform.
Dual Opteron 6376, 96 GB DDR3, Asus KGPE-D16, GTX 970
Mac Pro Dual Xeon E5645, 48 GB DDR3, GTX 770
Core i7 [email protected] Ghz, 32 GB DDR3, GA-X79-UP5-Wifi
Core i7 [email protected] Ghz, 16 GB DDR3, GTX 970, GA-X68XP-UD4
 
frumper15
Gerbil XP
Posts: 380
Joined: Mon Jan 18, 2010 3:25 pm

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 3:52 pm

I'm curious if your kind of work could also benefit from ECC RAM in the machine vs. regular DDR4 of the X99 platform. If you could benefit from the from ECC and like the idea of Dell support for a business application, I think the Precision 5810 workstation would fit the bill for you. I attempted to get CLOSE to the specs of the Maxwellator XL from the latest system guide as a point of reference. I put the lowest-end Quadro available in the Dell and the cheapest HDD option, but upped to the 6-core, 3.5GHz processor ( E5-1650 v3 ) and 4x4GB ECC memory and my total came out to $2,250 before tax and shipping. To that you would add an SSD for boot as well as the Titan X - to keep things simple I'll use the same 500GB MX200 from the system guide ($198) and the most popular Titan X from newegg ($1,040 shipped) you're looking at $3,488 all in (and you'll have an extra 500gb sata drive and low end quadro as well).

Using the system guide as a starting point we have the following: http://techreport.com/review/28621/the- ... -edition/9
$2711 all in
Subtract $1063 for the extra storage drives, sound card, and 980Ti for a $1,648 base to which you add $1040 for the Titan X for a new total of $2,688.

So, $800 gets you about the same speed machine in the dell with Xeon processor and ECC memory with better support vs. a machine that probably has better components, offers the ability to make some changes in your selections to better meet your needs, etc. I don't know your application and whether or not Xeon/ECC is a benefit, but you could just about buy a second Titan X for the price difference between the two options and if you're actually going to be putting those to full use that might make the decision all on its own for you.
i7-8086K | Z370 AORUS GAMING WIFI | 32GB DDR4-2400 | EVGA GTX 1080 Ti | 512GB 960 Pro | 27" Dell 2560x1440 Gsync | Fractal R6 | Seasonic Focus Plus 850W | Win10 Pro x64.
 
frumper15
Gerbil XP
Posts: 380
Joined: Mon Jan 18, 2010 3:25 pm

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 4:01 pm

Also in my digging I found you can still get a Titan Z for $1,600 which uses the older Kepler architecture but gives you 2707 GFLOPS of double precision vs. 192 GFLOPS for the Titan X. Quite the difference if that's what you're using. It's also got higher single precision performance, but with the smaller memory buffer per GPU it sounds like that wouldn't be attractive to you.
i7-8086K | Z370 AORUS GAMING WIFI | 32GB DDR4-2400 | EVGA GTX 1080 Ti | 512GB 960 Pro | 27" Dell 2560x1440 Gsync | Fractal R6 | Seasonic Focus Plus 850W | Win10 Pro x64.
 
DPete27
Grand Gerbil Poohbah
Posts: 3776
Joined: Wed Jan 26, 2011 12:50 pm
Location: Wisconsin, USA

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 4:03 pm

cobalt wrote:
If they have a model that you can configure with a high-end GPU, you can just configure it with a low-end GPU and stuff your own Titan X in it.

I would be VERY cautious about doing this. Dell isn't going to stuff a Titan-capable PSU into a workstation that's spec'd with a GTX 750.
Main: i5-3570K, ASRock Z77 Pro4-M, MSI RX480 8G, 500GB Crucial BX100, 2 TB Samsung EcoGreen F4, 16GB 1600MHz G.Skill @1.25V, EVGA 550-G2, Silverstone PS07B
HTPC: A8-5600K, MSI FM2-A75IA-E53, 4TB Seagate SSHD, 8GB 1866MHz G.Skill, Crosley D-25 Case Mod
 
BlackDove
Gerbil Elite
Posts: 694
Joined: Sat Oct 19, 2013 11:41 pm

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 7:00 pm

Ive been telling everyone to wait for Skylake LGA1151, but if youre getting a big chip youre going to be waiting a long time for Skylake-E. I would only wait if you decide to get an LGA1151 CPU.

Skylake Xeons will also introduce the Purley platform and there arent too many details yet. It will likely include an interface for XPoint memory on the E5 and E7 Xeons though. Maybe silicon photonics.

If youre doing a lot of SP work then you probably dont need to wait.

How much CPU do you really need? The only real difference between the 5960 and 5930 is two extra cores. Twice the price is absurd.

If you dont need multiple GPUs or a bunch of PCI-E SSDs then you might not even need X99 and you could get a Skylake LGA1151 i7 or E3 Xeon. Idk how much CPU you need though.

Whatever CPU choice you make, id add to the list an extremely high quality PSU like a Seasonic and an APC UPS thats big enough to support the whole thing under full load.

Also Quadro M6000 has no double precision. Youre paying for ECC GDDR5 and drivers. K6000, K40 and K80 are STILL their highest end GPUs.

If you need a lot of I/O performance, look into an Intel 750 Series NVMe SSD. If you decide to do that, X99 has more PCI-E lanes than LGA1151 will. If youre using SATA SSDs id go with Micron or Intel.

Id also recommend Crucial for whatever kind of RAM you need.
 
JustAnEngineer
Gerbil God
Posts: 19673
Joined: Sat Jan 26, 2002 7:00 pm
Location: The Heart of Dixie

Re: Building a Titan X based CUDA workstation

Thu Jul 30, 2015 7:45 pm

The NVidia reference cooler is pretty loud, but it is not as loud as my Radeon R9-290 with its reference cooler.
· R7-5800X, Liquid Freezer II 280, RoG Strix X570-E, 64GiB PC4-28800, Suprim Liquid RTX4090, 2TB SX8200Pro +4TB S860 +NAS, Define 7 Compact, Super Flower SF-1000F14TP, S3220DGF +32UD99, FC900R OE, DeathAdder2
 
cobalt
Gerbil First Class
Posts: 171
Joined: Mon Oct 30, 2006 11:28 am

Re: Building a Titan X based CUDA workstation

Fri Jul 31, 2015 9:30 am

DPete27 wrote:
cobalt wrote:
If they have a model that you can configure with a high-end GPU, you can just configure it with a low-end GPU and stuff your own Titan X in it.

I would be VERY cautious about doing this. Dell isn't going to stuff a Titan-capable PSU into a workstation that's spec'd with a GTX 750.


Yes, my point was to even have that option, you'll need to find a workstation which Dell will support with a power-hungry GPU. You're right, though, I neglected to mention that you need to make sure when you configure it with a different GPU that it's still got the good PSU. Should be possible to do, though maybe not through their web page.
 
deruberhanyok
Gerbil XP
Posts: 495
Joined: Sat Jul 17, 2004 9:30 am

Re: Building a Titan X based CUDA workstation

Fri Jul 31, 2015 10:24 am

They don't put low-end power supplies in the workstations, regardless of what video card goes in there. We order T7910s for work with little Quadro 310 cards in them for dual monitor output, and they ship with the same 1100W power supply that they would if we put K6000s in there.

Check when/if you order, obviously, but as far as I've seen, they have a standard power supply "size" for each model of workstation, and that's it.
<3 TR
 
bhtooefr
Lord High Gerbil
Posts: 8198
Joined: Mon Feb 16, 2004 11:20 am
Location: Newark, OH
Contact:

Re: Building a Titan X based CUDA workstation

Fri Aug 21, 2015 10:47 am

T5810 has both 425 and 685 W options, but it looks like you can specifically upgrade to a 685 W chassis for $45 without upgrading anything else.

Disclosure: I work for Dell, although I don't sell machines. (I'm in Services.)
Image
 
tftio
Gerbil In Training
Topic Author
Posts: 6
Joined: Thu Jul 30, 2015 1:33 pm

Re: Building a Titan X based CUDA workstation

Fri Aug 21, 2015 4:35 pm

Thanks everybody. I do deep learning CUDA(nn) programming, and I decided to build my own, so I went ahead and pulled the trigger on a dual Titan X/5930k machine.

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On