Nvidia confirmed that, as we suspected, the slowdowns are caused by a texture management issue, which would explain why 320MB cards are affected most often. The company says the root of the problem is applications failing to evict unneeded textures properly. The video card's drivers must account for this behavior and work around it.
Unfortunately, Nvidia also says the problem is more complex than it originally thought, so the fix won't be available until late August or early September. When it comes, the initial fix will only cover DirectX 9. A fix for DX10 will come later.