Looking for a PSU to Support a 2P, 4GPU Server

Enclosures, modding, blowholes, and the power needed to run it all.

Moderators: Nemesis, SpotTheCat

Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sun Nov 10, 2013 12:26 pm

I just realized that they may not make a PSU with enough wattage or connectors to drive a dual E5 system with 4 GPUs (possibly some of them having more than one GPU chip on board, such as with the HD 7990 or a possible future GTX 790).

Could it be possible that the PSU may put a cap on my wish to build a system with two CPU slots and four GPUs in a bench, tower, or free-standing form factor?

If you look at prices for all these parts, we're talking about a $5,000 to $6,000 system. I can't do that all at once. But I would like to do it in phases over the course of the next two or three years. Which means that the PSU (or multiple PSUs?) should be planned and sized early in the project so that I won't have to scramble when it's time to place an order.

For reference, this is going to be a high-compute machine for the great majority of time, and it will run F@H 24/7 except when I send it GPGPU work from my main workstation. It will probably live in the guest bedroom and will likely serve as a web browser when visitors come and stay; guests who don't arrive with their own laptop, that is... :P Heat generation will not be insignificant, so for comfort I plan to stop folding when guests are over.

It will be set up for Steam and therefore might be used for games on occasion, but probably not often; and certainly no need for Crossfire or SLI; just multiple badass GPUs for compute and one to drive a single big monitor.

Is this plan feasible with just one very big PSU, or am I getting into rarified air here? If multiple PSUs are needed, how do you do that and satisfy the 24-pin load on the second PSU...and where do you put a second PSU?

I also have a semi-related case topic. It was a judgement call, but I decided to separate the two.
BIF
Gerbil Jedi
Gold subscriber
 
 
Posts: 1546
Joined: Tue May 25, 2004 7:41 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sun Nov 10, 2013 3:36 pm

Well it is pretty simple.
LEPA G Series G1600-MA 1600W ATX12V / EPS12V

Is the largest PSU on the US market...there are a few 1500watt also.

However, trying to guess the wattage requirements for future graphics cards is harder.

but say 300 watt per card 4x and you are at 1200 watts.

running a 2p with 400 watts shouldn't be a problem.

Now, if your hypothetical future graphics card is OC and pulls 400 watts, then you need a second PSU. :wink:
mdk77777
Gerbil XP
 
Posts: 346
Joined: Fri Dec 12, 2008 3:42 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sun Nov 10, 2013 4:27 pm

No plans for overclocking, unless CPU/GPU are overclocked from the maker.
BIF
Gerbil Jedi
Gold subscriber
 
 
Posts: 1546
Joined: Tue May 25, 2004 7:41 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sun Nov 10, 2013 5:01 pm

Well, dual GPU cards do use more than average, but for FOLDING, I figure something between the average and Peak from Techpowerup.

http://www.techpowerup.com/reviews/NVID ... Ti/25.html

So you see, even the most power hungry dual GPU cards come in bellow 300 watts.

I would plan on only running the GPU client...you will need some cores from your CPU...and if you are running a total of 8 clients on a 4card dual GPU rig...you wouldn't have that much CPU left to run SMP anyway.

You could probably configure if you really wanted to push above 1600 watts, but there just wouldn't be much point in it from a PPD efficiency point of view.

Of course, this is all based on current points, and current graphics cards...handicapping the future is anyone's guess. :wink:
mdk77777
Gerbil XP
 
Posts: 346
Joined: Fri Dec 12, 2008 3:42 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sun Nov 10, 2013 5:17 pm

One other point:

I am concerned about your statement about heat in the guest bedroom.

You will be throwing off at least 800 watts folding 4 high end cards, even if they are not dual.

High ambient temps will rapidly age your cards unless you are planning on water-cooling.

Just saying, I hope you aren't planning on putting in the guest room and allowing the temps. to rise without cooling/ventilation.

800 to 1600 watts will heat a small house/appartment in the winter. :wink:
mdk77777
Gerbil XP
 
Posts: 346
Joined: Fri Dec 12, 2008 3:42 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sun Nov 10, 2013 5:33 pm

Good points!

The room I am thinking of gets no sunlight. The roof line above it and its only window both face North. The house air conditioning thermostat is in the family/greatroom where there are skylights and some windows that get sun, so that guest room is always the coolest in the house.

I'll have to monitor it of course, but if it overheats the room, then I'll have to find another place for it; possibly in the kitchen or main living area where the AC thermostat resides. If so, then there goes the guest room computer. I just have to be willing to be flexible.
BIF
Gerbil Jedi
Gold subscriber
 
 
Posts: 1546
Joined: Tue May 25, 2004 7:41 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sun Nov 10, 2013 5:38 pm

Also:

mdk77777 wrote:...I would plan on only running the GPU client...you will need some cores from your CPU...and if you are running a total of 8 clients on a 4card dual GPU rig...you wouldn't have that much CPU left to run SMP anyway...


Yes, that was the case for the older GPU work units, x16 and older. But the newer x17 WUs do not commandeer a whole CPU core. Edit: Hmmm, I think I need to get confirmation for the previous statement for both AMD and Nvidia.

Anyway, even if not true, the E5 Xeons I'm thinking of getting will have 8, 10, or 12 cores each (times 2 for hyperthreading and times 2 again for two processor slots), so CPU is not expected to be a constraint for folding in this system.
BIF
Gerbil Jedi
Gold subscriber
 
 
Posts: 1546
Joined: Tue May 25, 2004 7:41 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sun Nov 10, 2013 11:52 pm

so CPU is not expected to be a constraint for folding in this system.


No, I just was thinking about the max. possible load.

I run one GPU at around 200 Watt for 100K ppd.

Running an SMP in addition bumps up to 350 watts and and maybe another 10K ppd.

So for me, there is no good reason to run SMP.

The other big point are in ADVBIG units, but you need something like 16 CPU cores to meet the deadline...consequently, people who run 4P system run ADVBIG and not GPU...people who run 4 GPU, don't run SMP.

But some people just want to get the most points out of their rig, the deficiency and diminishing returns be danged!!

Anyway, realistically, I think you are good to go on the PSU, unless you go really, really extreme.

Good Luck. :wink:
mdk77777
Gerbil XP
 
Posts: 346
Joined: Fri Dec 12, 2008 3:42 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Mon Nov 11, 2013 12:15 am

I believe you should be good even with a 1200w PSU for something like, say, a 4-GPU folding setup. Just don't use CPU's for Folding (like someone else said - they are a waste of electricity), only Fold using GPU's. Which do not really consume much power - if you'll go with something like 4 Titan's (which are better for Compute tasks and specifically for Folding), each is rated at about 250W, but they do not really reach full load even when folding with x17 WU's (I notice more power consumption when playing games). It will still be consuming a lot of power, though, so for maximum efficiency you should be going with "80plus Platinum" - certified PSU and power your PC with a 220/230v outlet (if you primarily have 110/120v outlets at home).
My subscription allows you people to exist on this site and makes me a better human being than you'll ever be
JohnC
Gerbil Jedi
Gold subscriber
 
 
Posts: 1881
Joined: Fri Jan 28, 2011 2:08 pm
Location: NY/NJ/FL

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Mon Nov 11, 2013 3:21 am

JohnC wrote:...for maximum efficiency you should be going with "80plus Platinum" - certified PSU and power your PC with a 220/230v outlet (if you primarily have 110/120v outlets at home).


Oh boy, now I have to think about rewiring the house! :lol:

Thanks for the input; at least now I know the bad news first... :D

mdk77777 wrote:
No, I just was thinking about the max. possible load.
I run one GPU at around 200 Watt for 100K ppd.
Running an SMP in addition bumps up to 350 watts and and maybe another 10K ppd.

So for me, there is no good reason to run SMP.

The other big point are in ADVBIG units, but you need something like 16 CPU cores to meet the deadline...consequently, people who run 4P system run ADVBIG and not GPU...people who run 4 GPU, don't run SMP.


I follow you. Well, my eyes are bigger than my wallet, for sure, but surely a couple of 10 core/20 thread processors could do some major damage with BIGADV, yes? Until BIGADV is no longer available, that is (I've heard rumors).
BIF
Gerbil Jedi
Gold subscriber
 
 
Posts: 1546
Joined: Tue May 25, 2004 7:41 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Mon Nov 11, 2013 9:20 am

If you're worried about cooling the guest room or it becomes an issue in the future, you could add one of these pretty easily: http://www.amazon.com/Tripp-Lite-SRCOOL ... 002XITVCK/
i7-3770K | Asus P8Z77-V LK | 8GB DDR3-1600 | HD5850 | 128GB 840 Pro | Samsung F3 1TB | U2412M | Define R4 | Seasonic 520W M12II | Win7 Pro x64.
frumper15
Gerbil Team Leader
Silver subscriber
 
 
Posts: 238
Joined: Mon Jan 18, 2010 3:25 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Mon Nov 11, 2013 9:46 am

Yeah,

I was just advising against shutting the door to control noise...and then going to work/sleep.

8 hours of FOLDING in a confined room could get you up around 90 F+ really quick.
mdk77777
Gerbil XP
 
Posts: 346
Joined: Fri Dec 12, 2008 3:42 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Tue Nov 12, 2013 2:40 am

mdk77777 wrote:Yeah,

I was just advising against shutting the door to control noise...and then going to work/sleep.

8 hours of FOLDING in a confined room could get you up around 90 F+ really quick.


Yep, I leave the doors open so that the heat can have a chance to impact the whole house and thereby activate the house air conditioning more often. That room is not far from the aircon thermostat, so it would contribute as long as the door is left open.

frumper15 wrote:If you're worried about cooling the guest room or it becomes an issue in the future, you could add one of these pretty easily: http://www.amazon.com/Tripp-Lite-SRCOOL ... 002XITVCK/


The ad for that says it has a 71" tube. Really, six feet? I suppose that would allow you to put the unit someplace outside of the room you need to cool. I doubt I'll need one but it's good to have options!
BIF
Gerbil Jedi
Gold subscriber
 
 
Posts: 1546
Joined: Tue May 25, 2004 7:41 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Tue Nov 12, 2013 2:56 am

LOLz... Just crack open a window (or sliding door... whatever you have), you'll be fine. Your cards will most likely throttle down (especially if they have this appropriately named "Hawaii" GPU) long before it will get uncomfortable in the room :wink:
My subscription allows you people to exist on this site and makes me a better human being than you'll ever be
JohnC
Gerbil Jedi
Gold subscriber
 
 
Posts: 1881
Joined: Fri Jan 28, 2011 2:08 pm
Location: NY/NJ/FL

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Tue Nov 12, 2013 10:06 am

Your cards will most likely throttle down


Thanks Capt. Obvious...but if FOLDING, this is what you want to avoid.

PS...95 C would make the room pretty uncomfortable for me.

But you are correct, the system will self destruct long before that...the thing which we are planning to avoid. :wink:
mdk77777
Gerbil XP
 
Posts: 346
Joined: Fri Dec 12, 2008 3:42 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sat Nov 16, 2013 3:53 am

mdk77777 wrote:...the system will self destruct long before that...the thing which we are planning to avoid.


Yes, we do want to avoid self destruction!
BIF
Gerbil Jedi
Gold subscriber
 
 
Posts: 1546
Joined: Tue May 25, 2004 7:41 pm

Re: Looking for a PSU to Support a 2P, 4GPU Server

Postposted on Sat Nov 16, 2013 5:12 am

frumper15 wrote:If you're worried about cooling the guest room or it becomes an issue in the future, you could add one of these pretty easily: http://www.amazon.com/Tripp-Lite-SRCOOL ... 002XITVCK/


Over $600 for a 12,000 BTU Aircon?!?

The 5,000 BTU Frigidare I have works just fine, and for ~$120 to boot. (Per Frigidare's claims, good for up to 150sqft.)
PainIs4ThaWeak1
Gerbil
 
Posts: 48
Joined: Fri Jun 26, 2009 11:13 am


Return to Cases and Power Supplies

Who is online

Users browsing this forum: No registered users and 2 guests