How small do you need to go? mITX leads to a lot of trade offs in terms of heat dissipation, part compatibility, and cost. If you are looking to move to a smaller system but don't need to make it super small, often mATX ends up being the sweet spot. mITX is really best left to scenarios where you need to fit the system into a seriously space constrained setup.
The big thing you lose by dropping from the Z series to B and H series is the ability to overclock. If you are not overclocking you probably don't care -- just make sure that the new motherboard actually has any onboard components you rely on (e.g. if you have a lot of USB devices make sure it has enough ports for them all, if you are using onboard audio make sure the replacement motherboard has audio that meets your needs, etc.)
Main System: i5-4670K, ASUS Sabertooth Z97, 16 GB RAM, GTX 980 TI, 1 TB CRUCIAL MX500, Corsair 550D
HTPC: I5-4460, ASUS H97M-E, 8 GB RAM, GTX 970, CRUCIAL 256GB MX100, SILVERSTONE GD09B