Liquid cooling chills supercomputer, warms surrounding buildings

Thanks in part to the popularity of closed-loop water coolers, liquid cooling is becoming more and more prevalent in high-end desktop PCs. Servers are getting in on the action, too, although you won’t find them using the same cooling gear. IBM developed a custom liquid cooling system for SuperMUC, Europe’s most powerful supercomputer.

Cooling one or two components inside a PC is nothing compared to chilling a supercomputer with 18,000 Xeon processors. The accompanying 324TB of memory is also liquid-cooled. IBM refers to the cooling system as a “hot-water” design that can operate without the aid of a chiller or compressor. The water circulating through the plumbing gets up to 45°C, and it’s pumped through an exchanger that provides heat to the buildings surrounding the Leibniz Supercomputing Center where SuperMUC is housed.

According to IBM, using water cooling in this environment results in a 40% reduction in energy consumption versus a traditional air cooling. That’s good for savings of up to a million euros a year. Tree-huggers should be happy, too. SuperMUC is located in Germany, where all government-funded institutions are required to get their energy from sustainable sources.

IBM has been working with water cooling for decades, and it has interesting plans for the future. This presentation (PDF) on SuperMUC teases the potential of liquid cooling tucked between the layers of a 3D semiconductor. Keeping the liquid away from electrical interconnects is going to be challenging, but IBM already has a “test vehicle” for the technology.

Comments closed
    • TheMonkeyKing
    • 7 years ago

    45 degrees Celsius = 113 degrees Fahrenheit, not exactly “hot water.”

    But then again, I’m sure IBM marketing doesn’t want to sell anything related to “tepid.”

    • rrr
    • 7 years ago

    Maybe that could be used as “free” water heater for some residents or at least offices, etc.

    • Star Brood
    • 7 years ago

    I am in Germany and I love that there is so much emphasis on the environment here. Much more than in California where I am originally from.

    • Majiir Paktu
    • 7 years ago

    I rent a server from a data center, and I was a bit confused at first when the CPU temperature readout showed -1 C. At first I thought it was a sensor error, but it happily floated between -1 and 1. Seems my CPU is under ice water.

    • Bensam123
    • 7 years ago

    Cool idea… but if you spring a leak in this case, you don’t risk losing one computer. You risk losing everything in a rack, including complete data loss, and depending on how bad it is, adjacent racks too.

    Not just that, but if a leak happens, how do you replace it without shutting down all the electrical components around it? How do you locate it? Cleanup would be terrible and edge on the effects after fire suppression tech is used. Insurance wont cover something like this either.

      • shank15217
      • 7 years ago

      The water is probably de-ionized so it wont be conductive.

        • BloodSoul
        • 7 years ago

        ^THIS

        If you think they haven’t prepared for a leak on a project of this scale, you are quite silly.

          • cynan
          • 7 years ago

          Yes. I’m sure every leak prevention precaution was made.

          But I don’t see what they could possibly do if the system somehow did leak. Water is water. It is bad for electronics. I don’t see why they’re not using some sort of di-electric coolant (other than the fact that it wouldn’t have as much heat capacity as water and would have to be more aggressively cooled and not serve to heat the surrounding buildings as well).

          Perhaps the machines feeding coolant to each server rack can detect leaks (maybe via pressure sensors) and can shut down and isolate a single rack (both water flow and the computers themselves) in the event of a leak? This would still probably result in the loss of at least few server boards, depending how bad the leak was…

          Though probably highly unlikely, without further scrutiny, it still does sort of seem like an accident waiting to happen. Especially if they allow people into the server rooms. Humans = human error.

            • Bensam123
            • 7 years ago

            What I was thinking as well… Such a sensor would have to be super sensitive though and would probably end up tripping just from the system operating within normal tolerance. It only takes a drop or two of water to ruin a computer component.

            • BobbinThreadbare
            • 7 years ago

            Most likely the money they’re going to save on energy can be used to buy a few spare boards for the rare tiny leak.

            • Bensam123
            • 7 years ago

            I could see that… You can’t always perfectly diagnose something like this though. All it takes is a few drops some where that they can’t find or figure out and then they start having erratic issues that would be almost impossible to trace back to something. Most definitely it could start causing data corruption as well.

          • Bensam123
          • 7 years ago

          The titanic was unsinkable.

        • cynan
        • 7 years ago

        And just how long do you think the water will stay de-ionized once it’s exposed to air? I don’t think these server rooms are vacuums or filled with inert gases…

          • Bensam123
          • 7 years ago

          What I was thinking… Water doesn’t stay di-ionized forever. They would need a massive filtration system just to prevent leakage or wear inside the system from contaminating it. Yes, that’s minisicule, but it would eventually happen.

          Just as Cyann said, coming in contact with the air or the components it leaks onto as well as soaking into material like paper back capacitors. It would also form layers in between any two pieces of thin material, like a heatsink and stay there with very little evaporation.

      • rrr
      • 7 years ago

      Gee, I wonder how come no one at IBM thought of that. Your amazing insight should warrant you a job there…

      </sarcasm>

        • Bensam123
        • 7 years ago

        Your douchebaggery should warrant a special place on the internet too.

          • rrr
          • 7 years ago

          And here I am.

          You mad?

    • OneArmedScissor
    • 7 years ago

    Wow. The water gets considerably hotter than my CPU does with a stock aluminum heatsink. Do they also water cool the power supplies?

    I bet it really adds up to drop the northbridge(s) from the motherboard when data centers like this can move to Sandy Bridge E. Gourd only knows how much money it will save when Atom or ARM clusters can just have the RAM built in. The future may not be so wildly exciting for the average desktop, but it sure is for other things computers will be able to do for us.

    • MadManOriginal
    • 7 years ago

    What happens in the summer when they don’t want to dissipate the heat into another building?

      • Captain Ned
      • 7 years ago

      The heat energy can be used to drive an A/C chiller if necessary.

    • NeelyCam
    • 7 years ago

    This is exactly how things should be. It’s ridiculous to have a massive array of airconditioners to cool server farms when the housing next door has oil/gas heating.

      • ludi
      • 7 years ago

      For what it’s worth the core campus of my alma mater, the Colorado School of Mines is heated in the winter by waste heat piped over from the Coors brewing facility. They’re only about two miles away from each other at the closest corners, to the plumbing requirements are not onerous.

      There are rumors of a second pipeline that bulk-imports the finished product as well, but so far uncofirmed.

        • cynan
        • 7 years ago

        [quote<]There are rumors of a second pipeline that bulk-imports the finished product as well, but so far uncofirmed.[/quote<]. The only obvious issue with such a setup, of course, is that the "finished product" would be a Molson Coors one...

          • ludi
          • 7 years ago

          The target audience is primarily 18-22yo engineering students. Their tendency to be picky about beer selection is generally outweighed by their tendency to drink like fish on a thin budget.

          In any case, Killian’s Irish Red and Blue Moon are both tolerable products if there isn’t a local microbrew available, and those of us who actually live in the state can also obtain Colorado Native Lager.

          • Captain Ned
          • 7 years ago

          Since when did a college student care?

            • cynan
            • 7 years ago

            True enough. It’s just hard sometimes to pass up the opportunity to be a smartass – especially if it’s an elitist smartass!

    • Vulk
    • 7 years ago

    I got a glimpse of something like this at my former employer in 2009… It was impressive using a closed loop spray system that sprayed a non-conducting liquid directly onto the chips to be cooled, and watching it drip off the mobo into the collector was just… yeah.

    Using all that heat for building heating is just genius,and is the way into the future. It’s so much better to harness all that heat and just move it out of the room and dissipate it throughout the building, rather than simply running AC pumped in from the roof while running heaters everywhere else.

      • UberGerbil
      • 7 years ago

      [url=http://youtu.be/Tl_RB5Snhr4?t=10s<]Cray SV-2[/url<]? There was an older Cray design where they immersed the entire guts of the machine in a fluoridated liquid, which would bubble gently when it was thinking really hard

        • BobbinThreadbare
        • 7 years ago

        That’s awesome, it feels like 1970s sci-fi.

    • Dashak
    • 7 years ago

    It’s… beautiful.

      • BoBzeBuilder
      • 7 years ago

      And… huge.

        • rxc6
        • 7 years ago

        TWSS.

        • Arclight
        • 7 years ago

        You missed the part where the guy said they intend to make it a considerably smaller in a few years. Personally i think it was about time they used liquid cooling, but i can see why it took so long. After all they aren’t simply pumping the liquid to a radiator, rather they are letting the heat circulate inteligently to the heat exchanger and then recycle that heat for other purposes. Mad props to IBM

    • UberGerbil
    • 7 years ago

    The old Academic Computer Center at the University of Washington had to get a furnace installed when they retired their CDC 6000 in ~1990 because it had provided the only heat for the entire building.

Pin It on Pinterest

Share This