I think the progress has been more steady than it would seem by just thinking back a bit. DOS brought the first and second generation of true 3d acceleration at 16-bit color. Then came DirectX, and then 32-bit color, followed by steady resolution increases, antialiasing, anistropic filtering, steadily increasing environment sizes, higher and higher framerates and more resolution increases, hardware triangle setup, attempts to create games that actually mimic reality (many of the early 3d efforts were set in other environments because it's easier to make a "fictional" environment look alright since you don't have a "reality" to compare it to), hdr, fp16 color, increasingly complex post processing, physics processing (not necessarily 3d-accelerated, but has been in the past), 3d glasses, gp-gpu projects, high resolution textures, soft shadows, realistic looking water, tessellation, unprecedented performance with 120 hz monitors and massive levels of antialiasing that absorbs great amounts of power, better power per watt, better thermal profiles, more offloading from the cpu so that it can handle the much better ai routines, destructible environments (try just cause 2 for some destruction fun!), games of absolutely gigantic scale, and the list goes on.
I would agree that there are diminishing returns; each x% performance increase might only bring a steadily decreasing increase in performance. That said, I'd say things like resolution, fluid framerates, and antialiasing are quite important to realism. Even though you might not consciously see rendering artifacts, screen tearing, texture flickering, and aliasing, those do have a profound effect on the immersion level, and will have an ever increasing importance as we tackle things like "enough" video memory for textures that can be as detailed as our eyes can see, "enough" triangle throughput to give the designers pretty much total creative freedom, the power to render at very and sustainable high framerates with low latency, non-3d related items like realistic ai and accurate physics, etc.
I think ever since the first 3d cards came out, we've been working with diminishing returns; I doubt advances since the transition from 256-color 320x240 to 65,536-color 640x480 at astoundingly higher framerates will ever be topped. I think what's stagnating is our perception of the progress. If you told me 15 years ago that I'd have 2 video cards with 2 GB RAM each rated at 176 GB/s per card capable of rendering 28 Gpixels and 1.7 billion triangles with shader power measured in multiple teraflops per card, I would have wondered what on earth you could use that much power for.
It gets used.