Anatomy of a stutter
We have two directions we can go at this point: pursuing the multi-GPU micro-stuttering issue, and looking at cases where Fraps latency spikes aren't reflected as strongly in the frame delivery measured by FCAT. Let's start with the latter, and then we'll circle back to the micro-stuttering problem.
We're not trying to pick on the Radeon HD 7970 by using it as an example here. As you saw on the last page, each of the configs tested had one or more of these latency spikes during the duration of the test run. The 7970 plot and video just give us a nice test case for what happens when Fraps and FCAT measurements don't match.
We've zoomed in on the second of the two spikes in the 7970's Fraps plot. As you can see, there's a very small spike to 22 milliseconds in the FCAT plot, but a much larger spike, to nearly 50 ms, in the Fraps plot at the same spot. The question is: if frame delivery is still relatively even, as the FCAT results indicate, does it matter whether there was a spike in the Fraps data? As you might imagine, I was like a kid in a candy store when I got to pull up the video of the test run and see how it looked as I paged through the animation.
What I saw was... a quick but easily perceptible disruption in the animation, something much more on the order of the 50-ms delay indicated in the Fraps data. I've attempted to share this eureka moment with you by snipping out a brief video clip of the vaunted stutter in action. Since YouTube is gonna convert the video to 30 FPS no matter what, I took the liberty of slowing the source video to 15 FPS, in the hopes of keeping some semblance of each source frame intact. So hit play and buckle up. You should see a skip at about halfway through this six-second extravaganza.
Yeah, so YouTube pretty much adds its own stutter to the mix. Perhaps not the best tool for this job. Still, if you can get the video to play smoothly, the momentary stutter is real and perceptible. Although it's still pretty minor in the grand scheme, it confirms for me what Andrew Lauritzen has argued about the value of Fraps data. He was discussing a different Skyrim video of ours at the time, but the principle remains the same:
Note that what you are seeing are likely not changes in frame delivery to the display, but precisely the affect of the game adjusting how far it steps the simulation in time each frame. . . . A spike anywhere in the pipeline will cause the game to adjust the simulation time, which is pretty much guaranteed to produce jittery output. This is true even if frame delivery to the display (i.e. rendering pipeline output) remains buffered and consistent. i.e. it is never okay to see spikey output in frame latency graphs.
Even with buffering smoothing out frame delivery as measured by FCAT, the spike in the Fraps plot indicates a disruption in timing that has an impact on the content of the frames being displayed and thus on the smoothness of the animation.
We've been hearing an argument out of AMD about the value of Fraps data that I should address in this context. Folks I've talked to there have insisted to me that they've seen cases where a spike in Fraps frame times doesn't translate into an interruption in animation, seemingly casting aspersions on the value of Fraps data. After talking to AMD's David Nalasco last night, I think I understand this position better.
I believe Nalasco would modify Andrew Lauritzen's statement above to: "it is sometimes okay to see spiky output in frame latency graphs," simply because not every little hiccup or spike translates into a flaw in the animation that one can perceive. There are a couple of possible reasons why that could be the case. One has to do with the tricky question of what constitutes a stutter and what the threshold for human perception of a problem might be. Small interruptions may not matter if no one will notice them, especially with the display refresh cycle complicating how frames are presented. A related technical question is how large an interruption has to be in Fraps—by back-pressure in the rendering queue preventing the submission of a new frame—before a perceptible stutter is created. This issue is complicated by the varying ways in which game engines keep and advance the timing for their simulations. Some engines may be more tolerant of small bubbles in the pipeline if they advance time in regular intervals from frame to frame, even when those frames are being submitted closely together to refill the queue after a hiccup.
I won't argue with any of that. However, Nalasco also concedes that a sufficiently large frame time spike in Fraps will indeed translate into an interruption in the game's animation. I think he just wants folks to avoid obsessing over small spikes in frame time charts and to keep in mind that perception is the final arbiter of animation smoothness.
Which, you know, is why we create silly little videos like this one, examining a single case of a 50-millisecond frame and the visual interruption it appears to create.
However—and this is a huge caveat—we have some trepidation about declaring even this one particular example a definitive triumph for Fraps-based measurements. You see, like most folks who test gaming performance, we've removed the built-in frame rate cap in Skyrim. We already know that doing so causes some funky timing quirks for things like the game's AI, but it may also modify the game's fundamental timekeeping method for all of its physical simulation work. (The variable we've modified in order to "uncap" Skyrim is called "iPresentInterval", and we've changed it from "1" to "0." You may recall that Fraps measures when the game calls Present(). Hmm.) If our uncapping effort has changed the way time is kept in the game, it may have created the possibility of frame-to-frame timing issues that one would usually not see with the game engine's default timing method. This thought occurred to me on an airplane, on the way out to GDC, so I haven't been able to dig deeper into this issue yet. I definitely think it merits further investigation, and the frame-by-frame playback and analysis possible with the FCAT tool set should be a big help when the time comes.
|Radeon Pro Solid State Graphics keeps big data close to the GPU||11|
|Pascal powers up pro graphics with Nvidia's new Quadros||20|
|Phanteks breaks into custom liquid cooling with its Glacier G1080||11|
|Adata covers all of its bases with fast, durable external SSDs||0|
|MSI's Radeon RX 480 Gaming family will arrive in mid-August||13|
|Verizon nabs most of Yahoo for $4.8 billion||17|
|We beat the heat for food and fun at the "second-10th" TR BBQ||13|
|TR BBQ Day Shortbread||30|
|Watch the "second-10th" TR BBQ live in 360 degrees right now||12|