Single page Print

Testing Turing content-adaptive shading with Wolfenstein II: The New Colossus

Putting shader power where it's needed most

Nvidia's RTX ray-tracing acceleration stack might be getting the lion's share of press now that Battlefield V has helped debut the technology, but the Turing architecture is packed full of other fun tricks to help developers optimize the rendering performance of their applications. One of these is called variable-rate shading, or VRS. This Turing capability allows graphics cards built with that architecture to put their shading power where it's needed most in a frame, rather than shading every pixel naively. That naive approach might result in a graphics card "wasting" compute resources to arrive at the same result for each of a broad swath of uniformly colored pixels, for just one example, or working to shade a finely-detailed region of the screen that's then blurred by motion.

An illustration of the pixel groupings possible with Turing VRS, and how they might be applied to a scene. Source: Nvidia

To let shader power go where it's needed most, VRS allows developers to subdivide a 3D application's screen space into a grid of 16x16 pixel regions (among other possible division techniques, ranging from as fine as a per-triangle division or, more logically, per-object). According to Nvidia's SIGGRAPH presentation on the topic, each of those 16x16-pixel regions can, in turn, have one of several shading rates applied to it, from the naive 1x1 grid that would typically be used to shade every pixel in a scene today, to 2x2 groups of pixels for 1/4-rate shading, all the way out to a quite-coarse set of 4x4 grids for 1/16-rate shading. According to that presentation, then, the parts of a scene that need the most detail can get it via 1x1 shading, while those that are the most uniform or the most likely to be in motion can be shaded at the lowest rates to save shader horsepower for the parts of the scene that need it.

This approach could prove most useful for VR, where foveated rendering demands the ability to subdivide the scene into different regions of rendering resolution, but it has other applications in traditional 3D graphics, as well. VRS and the variety of subdivisions of a scene it allows for are the foundation for a technique called content-adaptive shading, or CAS. Here's how it works, according to the Nvidia Turing architecture white paper:

In Content Adaptive Shading, shading rate is simply lowered by considering factors like spatial and temporal (across frames) color coherence. The desired shading rate for different parts of the next frame to be rendered are computed in a post-processing step at the end of the current frame. If the amount of detail in a particular region was relatively low (sky or a flat wall etc.), then the shading rate can be locally lowered in the next frame. The output of the post-process analysis is a texture specifying a shading rate per 16 x 16 tile, and this texture is used to drive shading rate in the next frame. A developer can implement content-based shading rate reduction without modifying their existing pipeline, and with only small changes to their shaders.

Wolfenstein II: The New Colossus is the first game to implement CAS, and it is in fact the poster child for how the technique's analysis step works in the aforementioned white paper. Nvidia provides a visualized example of the result of its post-processing analysis on a frame taken from the game's good guys' submarine, Evas Hammer.

Source: Nvidia

In the figure above, the red squares represent the parts of the scene that the algorithm determined suitable for shading with the coarsest rates, while those with no color overlay get the finest 1x1 (or per-pixel) shading rate needed to reproduce fine detail or detail in motion. The radar and computer displays on the Evas Hammer are, from experience, rendered in motion, so it's no surprise that they get the highest shading rates in the example above.

Source: Nvidia

Despite the differences in shading rates applied to the various parts of a given scene by this technique, Nvidia believes it produces results that are practically equivalent to naively shading every pixel on screen. Wolfenstein II: The New Colossus just got a patch that enables CAS on Turing cards, so we can both test the performance of the technique for ourselves and see how it looks in practice.  Let's dive in.