David Kanter over at Real World Technologies has released his ruminations on the GPU portion of Apple's latest A10 SoCs. Apple has licensed Imagination Technologies' PowerVR graphics IP since the earliest days of the iPhone and iPad. However, after comparing developer documentation from both Apple and Imagination Technologies, Kanter concluded that Apple has been slowly replacing off-the-shelf components of PowerVR GPUs with its own proprietary designs. He argues those components have been appearing in Apple SoCs since the A8 chip that powers the iPhone 6 and 6 Plus.
Roughly speaking, Kanter describes a GPU as having three parts: fixed-function hardware, shader cores, and a software driver. The fixed-function hardware manages API commands and rasterization. The shader cores perform programmable graphics computation, and the driver translates API calls into commands for both the fixed-function hardware and the shaders. According to Kanter's research, the GPUs in Apple's A8 and newer SoCs employ still employ Imagination's fixed-function graphics hardware, but the shader cores and driver are unique to Apple.
Kanter speculates that a portion of Apple's GPU improvements in performance and power efficiency stem from the use of smaller half-precision floating-point registers. Half-precision is considered "good enough" for graphics, image processing, and machine learning. Kanter says that Apple's GPUs provide free conversion between different data types, enabling compilers and encouraging programmers to use "minimum data" when possible, rather than worrying about the computational cost of data type conversion.
Apple claims the "Portrait mode" feature on the iPhone 7 Plus uses machine learning to work its magic. Even if machine learning isn't involved, the faux-bokeh effects certainly involves the graphics and image processing points of the reduced-precision-math triangle. Kanter told us that many of Instagram's filters execute on the GPU for efficiency and performance reasons, too.
Kanter goes on to speculate about how Apple might benefit from exclusive GPU designs in their iOS devices. In-house GPU design might allows Apple to retain the benefits of research and develpment, rather than technology filtering back to competitors through Imagination Technologies. The article is a great read, and we are just skimming the surface. Go check it out here.