U.S. DoE to build fastest supercomputer with Tesla GPUs

Talk about some nice PR for Nvidia. The U.S. Department of Energy is apparently planning to deploy the world’s most powerful supercomputer by a long shot, and it’s going to use Nvidia’s Tesla GPUs to do it.

According to Nvidia, the supercomputer will be deployed at the Oak Ridge National Laboratory in Tennessee. Titan, as it’s named, is meant to have peak floating-point computing performance in excess of 20 petaFLOPS. That would make it over twice as fast as the world’s current fastest supercomputer, Japan’s K computer, which tops out at 8.16 petaFLOPS. Nvidia claims Titan will be “three times more energy efficient,” as well.

Here are all the dirty details about the deployment:

In the first phase of the Titan deployment, which is currently underway, Oak Ridge will upgrade its existing Jaguar supercomputer with 960 Tesla M2090 GPUs based on the NVIDIA® “Fermi” architecture. These GPUs will serve as companion processors to multi-core CPUs in this Cray XK6 supercomputer. In the second phase, expected to begin in 2012, Oak Ridge plans to deploy up to 18,000 Tesla GPUs based on the next-generation architecture code-named “Kepler.”

Nvidia doesn’t say whether the plan is for Titan to become sentient, although it does note the supercomputer will be used “in a broad range of fields, including material science, energy technology, medical research, geoscience, and others.”

Comments closed
    • ronch
    • 8 years ago

    Opteron 6200 series processors based on BD should fit right in with these kinds of systems. I wonder how much of the total computing power is attributable to Opteron and how much to Tesla.

    • Coran Fixx
    • 8 years ago

    Will be built and then will be revealed as Homeland Security Skynet. Will pierce tinfoil.

    • xiaomim
    • 8 years ago
    • anotherengineer
    • 8 years ago

    I wonder how much the DoE will be paying for each Tesla GPU vs the equivalent consumer gamer card.

    I remember people laughing at fermi when it came out, but this is what it was truely designed for, number crunching for Gov, business and industry, where the big money is.

      • sschaem
      • 8 years ago

      Those card cost ~$4000 , I’m sure Cray is getting them at a HUGE discount. Same thing with Opteron processor.
      20 petaflop represent 20,000 teraflop. a single fermi is 1teraflop. So for 60million they get 20,000 fermi + 20,000 opteron

      So ~$3,000 for a fermi + interlagos (much less when you account for all the ram, cabinet, boards, etc..)

      Cray is getting a great deal, way over 50% below retail. Unless my math is not correct ?

        • d0g_p00p
        • 8 years ago

        Cray does not buy boards only chips. So they are getting them at a very cheap price.

          • sschaem
          • 8 years ago

          Yes, that what I just said. Thank you for confirming with just more speculation and no facts.

    • shaq_mobile
    • 8 years ago

    But can it run Crysis?

      • A_Pickle
      • 8 years ago

      On highest, at long, long last.

      Too bad it doesn’t get EyeFinity. 😀

        • Goty
        • 8 years ago

        It doesn’t get any monitors at all! No display controllers on Tesla 😉

    • tviceman
    • 8 years ago

    I thought Charlie said Oak Ridge canned Nvidia…. I wonder how he’ll spin this now.

      • NeelyCam
      • 8 years ago

      Maybe he’ll be on a “Quiet Period”

        • HighTech4US2
        • 8 years ago

        Kind of like right now.

        With bulldozer being released tomorrow he is conveniently out of the office and somehow, even with this new fangled thing called the internet, he can’t communicate about any of the leaked benchmarks.

        Again, you only here him when and if he is right or when he spews FUD at Nvidia. If he ever is wrong he can’t be found and/or is mute.

          • NeelyCam
          • 8 years ago

          He already said he won’t leak any more BD stuff out of respect for ‘NDAs and other journos’

            • HighTech4US2
            • 8 years ago

            Since when has he ever signed a NDA?
            Or even honored one.

            He always puffed out his chest and belched out “I don’t sign no effing NDA’s”.

            Oh, and now he will “respect” AMD’s NDA. Yea he will when he knows the result is s – hit

      • HighTech4US2
      • 8 years ago

      Which is why you should take what charlie says about Nvidia as not gospel.

      He can be wrong. More so when Nvidia is involved.

    • Ryhadar
    • 8 years ago

    [quote<]The U.S. Department of Energy...[/quote<] [quote<]...based on the NVIDIA® [b<]"Fermi"[/b<] architecture[/quote<] There's a joke to be made here about Fermi's power consumption. I just know it. 😉

      • HighTech4US2
      • 8 years ago

      Yes the joke is on charlie and others who follow him blindly have claimed that Nvidia’s lost this account because of Power requirements and Heat Generated.

      “That would make it over twice as fast as the world’s current fastest supercomputer, Japan’s K computer, which tops out at 8.16 petaFLOPS. Nvidia claims Titan will be “three times more energy efficient,” as well.”

      Please note that even with the current Fermi Teslas it has been well documented that Super Computers with them do save substantial power over just using massive quantities of CPU’s.

    • sschaem
    • 8 years ago

    The fine prints:

    “Factors that could affect actual future events or results include, but are not limited to, the risk that the XK6 system upgrades required by ORNL are not delivered in a timely fashion or do not perform as expected, the risk that the AMD “Interlagos” processor, shipments of which began later than originally anticipated, is not available to Cray in the quantities necessary when needed,”

    I guess thats why AMD doesn’t want to blow their horn on this one and instead show some video stitching software on their front page.

    • Tristan
    • 8 years ago

    Do not use Tesla. Use APU…
    Very cheap, integrated CPU…
    Again, world against AMD

      • A_Pickle
      • 8 years ago

      Said the armchair expert.

      I love the APU’s as much as the next man, but if you think these are chips suited to supercomputers, you’re smoking something [i<]ridiculous[/i<].

      • NeelyCam
      • 8 years ago

      I don’t usually like poetry but this was rather enjoyable.

    • Captain Ned
    • 8 years ago

    Oak Ridge?

    This beastie will be used to simulate nuclear explosions in an effort to “validate” the current stockpile without actually setting one off.

      • cobalt
      • 8 years ago

      No. You’re thinking of classified systems at some other national labs, like Lawrence Livermore, Los Alamos, or Sandia. Many DOE national labs are generally open science — this is an unclassified machine. You can browse the project list: [url<]http://www.olcf.ornl.gov/leadership-science/[/url<]

        • Ragnar Dan
        • 8 years ago

        If memory serves, wasn’t Oak Ridge lab where the contents of the “Libyan” nuclear weapons R&D center (filled with Iraqi scientists and engineers, and evidently not discovered/scanned by American satellites, while putatively being less than a year from their first weapon test) moved to in late 2003? If that is so, then they have at least some secrets the govt. wants kept (except when idiots like Bill Clinton’s first Energy Sec’y allowed the publication of nuclear weapons designs on their public servers, luckily before most of the world knew about or how to read it and before the govt. used http on their servers, but North Korea indeed did have knowledge of how to copy from their servers, as likely other intelligent foreign governments/entities did, too).

        (Please pardon the rant, but some facts need to be known by more people.)

    • sschaem
    • 8 years ago

    Nice for AMD, this is a high profile user… I wonder if AMD marketing department will think of using this to change bulldozer image.

      • StuG
      • 8 years ago

      Where did you find that AMD was also in on this?

        • Goty
        • 8 years ago

        If the nodes are based off of the Cray XE6, then it uses Opterons.

        • sschaem
        • 8 years ago

        [url<]http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsArticle&ID=1615561&highlight=[/url<] "The first phase of the contract will include replacing the Cray XT5 compute blades with Cray XK6 compute blades, which will feature the upcoming AMD (NYSE: AMD) Opteron(TM) processors code-named "Interlagos" Notice "Upcoming" .. sigh

    • quasi_accurate
    • 8 years ago

    How come no mention of the fact that Cray XK6 uses AMD Opteron alongside the Teslas?

      • Waco
      • 8 years ago

      On top of that, it’ll probably be using Interlagos. None of the news sites mention this though…

        • NeelyCam
        • 8 years ago

        It is Interlagos:

        [url<]http://www.olcf.ornl.gov/2011/10/11/oak-ridge-national-laboratory-awards-contract-to-cray-for-%e2%80%98titan%e2%80%99-supercomputer/[/url<]

      • OneArmedScissor
      • 8 years ago

      Nvidia PR is rabid.

      AMD PR is…do they have a PR department?!?

        • sweatshopking
        • 8 years ago

        yeah. i’m pretty much it. Between working here, and working for AMD, I still have time for games!

        You should probably buy intel.

    • Goty
    • 8 years ago

    20 PetaFLOPS and they STILL won’t give you enough damn time on the thing to get anything useful done with it.

      • willmore
      • 8 years ago

      In their defence, your grant proposal should have said something more scientifically valid than just “Can it play Crysis?”

        • Goty
        • 8 years ago

        I specifically noted that I’d only need the whole machine long enough to get bored with BF3 on its highest settings. I don’t see what’s so unreasonable about that.

          • willmore
          • 8 years ago

          You got a while 5 minutes on it? Wow.

    • Arclight
    • 8 years ago

    And one year after they finish it, the next gen graphics will be used to outclass this project. At this point i don’t even care how powerfull it is, i’m just glad they keep doing it for research purposes.

      • sschaem
      • 8 years ago

      And it will be a drop in replacement… like the next gen 20core bulldozer.

    • dpaus
    • 8 years ago

    I thought the project was code-named “[url=http://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project<]Colossus[/url<]"??

      • willmore
      • 8 years ago

      You’re confusing this with the Forbin Project.

        • dpaus
        • 8 years ago

        They’re both burried thousands of feet below the surface of the earth, under an imprenetrable, impervious mountain, no? (i.e., presumably in the cubical next to AMD’s PR Department)

          • willmore
          • 8 years ago

          Here is your internet, sir. You earned it.

    • jdaven
    • 8 years ago

    What no intel inside?

      • Farting Bob
      • 8 years ago

      Nope, AMd CPU’s and Nvidia GPU’s for this one! Are the CPU’s used of the new bulldozer varient or still on the previous architecture?

        • A_Pickle
        • 8 years ago

        Bulldozer. Opteron 6200-series “Interlagos” 16-core processors. 😀

          • NeelyCam
          • 8 years ago

          [quote<]"The initial system upgrade will replace all of the processors with the latest AMD Opteron processors codenamed “Interlagos,” taking the system from two 6-core processors per node to one 16-core processor per node, and facilitate the addition of GPUs."[/quote<] So, they 'upgraded' from 2x6core nodes to 1x8-module nodes...

            • sschaem
            • 8 years ago

            2×4 module.

            The upgrade is not obvious, just looking at the execution unit available… Until we get power and refined benchmark its hard to tell if its a minor upgrade or huge (mainly compute/power ratio). The fermi chips will be doing the heavy lifting, the opteron will act more often then not as support chips.

    • sweatshopking
    • 8 years ago

    yay?

Pin It on Pinterest

Share This