Moore's Law is about transistor scaling, not clock speed scaling.
You can use extra transistors to enable clock speed scaling, but recently, architects have instead generally sought to improve:
-Instruction-level parallelism (IPC increases)
-Thread-level parallelism (SMT and core count increases, along with larger last-level caches for better sharing)
-Integration (Cost reduction, power reduction, some I/O latency reduction)
-Dynamic voltage and frequency scaling (to get the most clock speed and perf out of a given power budget)
The first category, TLP/IPC increases, has been the source of much of the incremental goodness in Sandy Bridge, Ivy, and Haswell. You can gain a lot of performance by dedicating more logic to improving branch prediction accuracy and such.
But so have each of the others. No front has been neglected, really, in big x86 chips in recent years.
The interesting thing now is that two limitations are becoming more important, even as transistor scaling promises to continue for a while (probably).
One, power is the primary performance constraint for CPUs, bar none. That's why CPU architects moved away from clock speed scaling as a primary tool, and each decision about any of the other ways of improving performance is made with an eye toward power efficiency.
Two, the goodness of Moore's Law seems to be waning as the benefits of pushing into higher gate densities fade. Taking photolithography further now means using double-patterning (dual masks with an offset) on more metal layers on a chip, for example. Double-patterning costs more, and it means a die shrink won't necessarily give you more transistors for "free." Similarly, some process shrinks have had little to no (or negative) benefits for power consumption lately. FinFETs are helping, but things only get harder going forward.
Future CPU architectures will have to be built within these constraints. We could see new architectures coming out more slowly as a result, but I wouldn't count on it. Instead, we see to be seeing an explosion of new designs custom-tailored for specific market segments and missions, both in the x86 world and more notably in ARM.
Scott Wasson - "Damage"
Editor - The Tech Report