The world of big iron computing seems to be laser-focused on machine learning these days. Whether it's graphics chip makers Nvidia and AMD producing silicon exclusively for machine learning, or search engine Bing using custom FPGAs to accelerate repetitive mathematical operations, every technology company these days seems to have an AI accelerator strategy.
Google isn't resting on its laurels, either. At its I/O conference today, the company introduced a second-generation version of its Tensor Processor Unit from a year ago, called the Cloud TPU. Google can combine multiple Cloud TPUs into four-chip clusters, and the cluster pictured below offers up to a claimed 180 TFLOPS of floating-point capability. The search giant didn't say whether that's for FP16 or FP32 math, but given the hardware's focus on machine learning tasks, that figure surely refers to reduced-precision number crunching. For comparison, Nvidia's just-introduced Tesla V100 accelerator leans on dedicated tensor hardware to provide 120 machine-learning TFLOPS.
Picture courtesy TR's anonymous Google I/O correspondent
The new TPUs can be assembled into what Google calls pods. Each pod contains 64 second-generation TPUs and should be good for an aggregate 11.5 PFLOPS of compute power. The previous TPU was used for inference (execution tasks) only, but Google says the new TPU can be used for both training and inference tasks. As an example, the company says that a training task that required a full day for "32 of the best commercially-available GPUs" can be done in an afternoon on eight of the new TPUs.
Google will be selling access to its TPUs through its Google Cloud Compute platform, where virtual machines can be used to group TPUs, Nvidia GPUs, and Intel CPU cores together as required to run the TensorFlow machine-learning framework. Gerbils who have used Shazam to identify a music track recently may have had results provided by a Nvidia GPU that is part of Google's Compute Cloud. While the majority of access to the Cloud Compute platform will be paid, the company will provide access to 1000 TPUs to machine-learning researchers as part of its TensorFlow Research Cloud.